ZooData
The data layer for AI agents
833 followers
The data layer for AI agents
833 followers
ZooData turns any URL into agent-ready JSON, so AI agents can work with structured data instead of raw HTML or bloated markdown. Use ~75% fewer LLM tokens, pay only for the fields you use, and skip extra extraction credits.Beyond extraction, ZooData gives agents pre-analyzed e-commerce intelligence — competitor, market, traffic, and consumer insights — live for Amazon and TikTok. API, CLI, and MCP server included. Start with 1,000 free credits, no card required.
curious how this holds up against anti-bot layers - a lot of the messiest sites to scrape aren't messy because of markup, they're messy because they actively try to block automated requests. does the per-field pricing still apply if a site just serves you a captcha wall instead of html, or is that a separate failure case
ZooData
@omri_ben_shoham1 Fair question, straight answer: a captcha wall / block = a failure case, not billed — credits are only consumed when a call successfully returns data, and the status code in the response is the upstream's real one, so a block is never disguised as a success. The anti-bot arms race is our infrastructure's job (rendering, retries, network diversity), but we won't claim 100% — throw your nastiest site at it; if it fails, it costs you nothing.
the field-level trust discussion above is the real meat of this thread. one I didn't see covered: what happens on a listing/search-results page that needs pagination or infinite scroll to surface everything - does a single call return just what's in the initial DOM, or does ZooData drive the scroll/pagination itself to assemble the full result set before handing back JSON?
ZooData
@galdayan One call = one rendered page-view — we deliberately don't drive scroll/pagination behind a flat price (that's a cost black box, against everything in this thread). For full result sets: infinite scroll → the interactive variant (you pass explicit scroll/click/wait actions, hard-capped cumulative time, bounded cost); classic pagination → your agent loops over page URLs, flat price each, linear spend. Your agent owns the depth decision; we keep each unit deterministic.
@kyle_dong That's a clean split - keeps the flat-price unit honest instead of hiding variable cost inside it. Makes sense that the agent should own the depth decision rather than you guessing at it server-side. Good answer, thanks for walking through both paths.
ZooData
@galdayan Thanks! You summed it up better than we did: primitives stay honest, orchestration belongs to the agent. We keep the first half true — the second half is your stage. Come back and show us what you build with it.
@kyle_dong will do, bookmarking this one to try once I've got a real scrape target to point it at.
The 'any URL to structured JSON, pay only for the fields you use' angle is what makes this actually fit an agent loop instead of burning tokens filtering markdown. When I specify the fields, is that a per-request schema I define, so the same URL can return different shapes for different agents, or a fixed schema inferred per domain? And on JS-heavy pages that lazy-load content, does extraction run against the fully rendered DOM or the initial HTML?
ZooData
@hazy0 Two good questions, straight answers:
1.The schema is fixed per page type — not per request, not per domain. The system auto-detects the page type (product / article / listing / entity …), and each type has a fixed field set — so the same URL returns the same shape to every agent: deterministic, predictable, cacheable. Also a quick calibration on "pay only for the fields you use": pricing is flat per call; the field economics live in the response containing only meaningful fields — no boilerplate, no noise, your context only ingests signal. There's no caller-supplied custom schema today — you're the second person in this thread to ask, and we hear the signal 😄
2.The fully rendered DOM. Fetching runs in real browsers with JS executed — content that isn't in the initial HTML but appears after render is visible to extraction. For content that only appears after interaction (click-to-expand and friends), there's a separate interactive variant in the API for exactly that.
And honestly, "same URL, constant shape" is an asset in an agent loop: the second time your agent hits the same kind of page, the shape is identical to the first — no defensive programming required.
@kyle_dong The fixed-per-page-type schema plus a separate interactive variant is a cleaner split than I expected; deterministic shape really does matter more than flexibility in an agent loop. The one thing I would want to pin down: when auto-detection guesses the page type wrong (a product page that is really a listing, or a hybrid), do I get a confidence signal or a way to force the type, or just a confidently-wrong shape with no flag? That is the failure mode that would bite hardest in an unattended loop.
ZooData
@hazy0 We know that fear well — in an unattended loop, the scary thing was never the error, it's being wrong with a straight face. Good news: that one doesn't happen here, because the page_type we decide on is echoed back in every response — the guess is never a black box. Expecting product and got listing? One comparison, and your loop branches on the spot. As for a confidence score or forcing the type manually — honestly, neither exists today. But a wrong-type extraction tends to give itself away anyway: the wrong schema can't fill its own core fields, so what comes back is a pile of absences, not a plump wrong shape. So the flag you're asking for is already there — the classification is always visible; just have your loop check it.
@Kyle Dong that's the right answer honestly, "we don't disguise a block as a success" is the thing that actually matters, most tools I've tried just eat the failure silently and you don't find out until your data looks wrong three steps downstream. does the response include which failure mode it hit, like captcha vs rate limit vs structure change, or is it just a generic fail code?
ZooData
@omri_ben_shoham1 Finer than a generic fail code — but honestly, not a full forensic report either. The design is neutral classified codes + real status codes:
- Blocked / access denied (captcha walls fall here) → an access-denied class error, with the target's real HTTP status in meta.statusCode — never one we made up;
- Rate limiting on our side → you get an explicit 429 with X-RateLimit headers before any work runs — and nothing ran, so nothing's billed;
- Structure change → that one's actually not an error: the page is still there and readable, so markdown comes back as normal — only the structured json goes absent or degraded. You get an honest less, never a silent wrong. Which loops right back into the field-trust discussion above.
As for why we don't put finer anti-bot attribution (exactly whose wall, which signal) in the response: two reasons. One, those internal signals change shape and name — letting you integrate against them is handing you a landmine. Two, frankly, precise block-detection details published in an API mostly benefit the wrong audience. We watch the full attribution on our internal dashboards — its job is for us to go fix things, not for you to triage.
What your agent actually needs collapses to three lanes: retryable (transient upstream) / don't-retry (access refused) / content-ok-but-structure-degraded. All three are distinguishable from the response.
Structured input is a better foundation for agents than making the model repeatedly interpret noisy pages. How do you preserve context that doesn’t fit neatly into the extracted schema but might become important to the agent later?
ZooData
@amir_mehrabi Great question — and it's exactly why formats are composable on our side. A single call can return json and markdown together (add rawHtml if you want full fidelity): json is for acting now, markdown is for "might matter later." The markdown is a page-faithful rendering of the full content — the long tail the schema didn't catch (a footnote, a promo clause, some odd badge) is still in there, not thrown away.
The agent pattern we see looks like this: act on the fields immediately; stash the markdown cheaply outside the context window (it's just text); and the day a decision actually needs the long tail, feed that one page back to the model to read once. That's the full meaning of "structured-first" — not structure-only, but turning the model's prose-interpretation from "re-read on every hop" into "read once, on demand." What we eliminate is repeated interpretation, not information.
And a footnote: requesting an extra format in the same call doesn't change the price — one call, one price. Keeping that "might matter later" escape hatch costs you nothing at the margin.
The 75% token reduction claim makes sense structurally but I'm curious how it holds on pages where the most valuable data is buried in dynamic elements that only render after JavaScript executes , things like live pricing widgets, inventory counters, or seller ratings that load asynchronously. Is the JSON output pulling from the fully rendered DOM or the initial HTML response, because for e-commerce intelligence that difference is where the real data quality gap tends to sit.
ZooData
@jasnoor_singh_oberoi You've asked at exactly the right layer — two-part answer:
1.General extraction (any URL): it runs against the fully rendered DOM. Fetching executes in real browsers — JS runs, async widgets render, then extraction happens. Content that's absent from the initial HTML but appears after render is visible to us.
2.E-commerce intelligence endpoints: the three things you listed are literally the product's core fields. Displayed price, inventory quantity, seller/shop ratings — on our realtime product endpoints these are first-class typed fields, part of the contract, not lucky catches from prose. What comes back is the page's real displayed state at collection time. A live example: when a page shows an asterisked promo price ("$1*"), that's exactly the raw state we return — rather than inventing a number. That brand of honesty has come up a few times in this thread.
As for how this relates to the 75%: rendering completeness and token savings are orthogonal — rendering determines what exists to structure; structuring determines how much your context ingests. We render the page fully first, then compress it into fields. Doing it the other way — grabbing initial HTML to "save" — isn't saving tokens, it's losing data. You're right that this is where the e-commerce quality gap lives.
@kyle_dong The separation between rendering completeness and token savings is a clean way to frame it, rendering determines what exists, structuring determines how much your agent ingests. The asterisked promo price example is the right kind of honesty too, returning the raw displayed state rather than interpreting it means the agent makes the call on what that price actually means rather than having the extractor paper over ambiguity. That's the correct design for anything downstream that acts on the data.
ZooData
@jasnoor_singh_oberoi Thanks! You've nailed the division of labor — we'd only add the last half-sentence: an extractor that interprets for you is making your agent's decision for it — silently. We keep the fidelity; the judgment stays yours. If you ever catch a field crossing that line, tell us — that's a bug
neat framing, agent-ready JSON instead of raw markdown. when a site restructures its page layout, does the schema auto-remap to catch the drift or does it silently break until someone notices the fields are empty?
ZooData
@sabber_ahamed Neither 😄 The real answer: detected on the spot, degraded honestly, re-learned automatically. Every extraction passes core-field validation — the moment a redesign breaks structure, validation fails and the result is never stamped success; unreadable fields are absent (absence ≡ not extracted), never invented. The structure then re-learns the new layout — first pass slower, speed recovers, self-healing with zero action from you. And every drift trigger lands with attribution on our dashboards — "noticing" is our job, not your downstream data's.