Building an API that pulls data across 13 social platforms, and the thing that's eaten the most engineering time isn't the scraping itself, it's how differently every platform breaks. Rate limits that change without notice, auth flows that expire silently, structured data one week and a wall of JS the next.
Curious what's bitten other builders here. Which platform has been the worst to depend on, and what did you end up doing about it?
Social Fetch
Hey Product Hunt 👋 Luke here.
I built Social Fetch because I kept running into the same problem.
Every project that needed public social media data started out simple enough. Grab some YouTube data. Add Twitter. Then someone asks for TikTok. Then LinkedIn. Then Reddit.
Before long, I wasn't building the product anymore, I was maintaining scrapers.
Platforms change. Formats are all different. Something is always breaking.
It felt like a solved problem that nobody had actually solved.
So I spent the last couple of years building Social Fetch: one API for live public data across 20+ platforms.
What surprised me wasn't that people wanted social data. It was how many different things they were building with it. AI agents. Creator tools. Lead generation. Brand monitoring. Research. Things I never would have thought of myself.
One thing I care a lot about is keeping it simple: • one API • live data • consistent responses • subscriptions + pay-as-you-go credits that never expire
I'd love to know what you'd build with it.
Or if you've ever built your own scraper... what finally made you regret it? 😅
I'll be hanging around all day if anyone wants to chat or has questions.
— Luke
RiteKit Company Logo API
@lukem121 Congrats on the Social Fetch launch — one clean API for social data across every platform, with a request ID on every call, is exactly the boring-in-a-good-way that people building on social actually need. We made you a free launch video (below): launches with a video tend to do better, so use it on your Product Hunt page, embed it, or share it anywhere — yours to keep, 100% whitelabel, no strings.
We built it with FoxPlug (https://foxplug.com) — paste your site and it turns what you just shipped into a launch video, images, and posts in about 30 seconds. This one is on us.
@lukem121 +1 to your forum thread on which platforms break the worst — the silent auth expiry and "structured one week, wall of JS the next" flip is exactly the tax that makes people regret rolling their own.
Before wiring this into an agent I'd want to know: is "live data" fetched synchronously per call, and what's the rough p95 when a platform is being slow? And when something breaks on your side, does the API return a typed error / degraded flag so the caller can tell "no results" from "scraper down"?
Also curious whether there's an MCP server yet, or REST-only for now. 👌
Social Fetch
@akbar_b Appreciate the read on the forum thread, and yes, that flip between clean structured data and silent breakage is exactly what pushed us to build this.
To your questions: fetch is synchronous per call, live data every time, no caching, averages around 3.2s. We track average response time but haven't published p95 yet, can follow up with a real number if that matters for your use case.
On errors, we distinguish these on purpose: not_found means a real lookup ran and the target genuinely doesn't exist (still charged, it's real work). lookup_failed and 503 temporarily_unavailable mean something broke on our side, and those aren't charged. So you can tell "no results" from "scraper's having a bad day" without guessing.
And yes, MCP server just shipped, 150+ tools, works with Cursor and Claude out of the box.
Hooked it up to a small TikTok enrichment flow this morning and it just worked, no schema headaches or weird rate limits. The pay-as-you-go credits make it easy to test without committing to a monthly plan.
Social Fetch
@bekirblxx Glad it slotted in clean. That's the goal, one less integration to babysit.
How does the pay-as-you-go pricing actually work for someone pulling data from multiple platforms at once? Like is each API call priced the same across TikTok, Instagram, and LinkedIn, or does it vary by endpoint and data type?
Social Fetch
@gllatalarnm2d Good question. Pricing is flat across almost every endpoint, one credit per call, TikTok, Instagram, YouTube, X, whatever mix you're pulling. LinkedIn's the one exception at 2 credits, since that data's more work to fetch reliably. Everything else is uniform.
How does it handle rate limits across platforms like TikTok and Instagram when you're pulling a high volume of profiles and posts at the same time?
Social Fetch
@doanlwbr No enforced rate limits or RPS caps on the metered routes, your limit is really your credit balance rather than a request ceiling. We'd suggest staying under about 500 concurrent requests for best reliability. But you're more than welcome to push it to its limits 💪🏻
Plugged in the YouTube endpoint yesterday and got back transcript plus engagement metrics in one call, which saved me from gluing together two separate tools I was already paying for. Pricing per request makes a lot of sense for my spiky usage too.
Social Fetch
@recepedtx That's exactly the use case we built the combined transcript+engagement call for, appreciate you flagging it saved you a second tool.
Congrats on the launch, Luke. this is painfully real - the maintenance tax on multi-platform is the whole game.
Two things I'd genuinely want to know before wiring this into an agent:
-When a platform flips (TikTok especially), does the API return a typed error / degraded flag so the caller can tell "no results" from "scraper down"? For anything automated that distinction matters more than raw uptime.
- On the YouTube transcript endpoint - is that pulled live per call or cached, and rough p95 when the platform's being slow?
Either way, nailing one consistent response shape across 20+ platforms is the hard part and clearly what you obsessed over. Nice work.