Launched this week
webatla
The global domain intelligence database.
7 followers
The global domain intelligence database.
7 followers
Download the global domains database — 406M+ active domains with DNS, WHOIS, technology stack and ranking, as structured JSONL. Fresh daily export, pay once, no subscription.
How fresh is "fresh daily" actually—does the export drop at a set UTC time, and would I get notified when a new file is ready? Also wondering if older snapshots are kept around for diffing.
@ogluzafer19806 Hello Zafer,
Good questions.
We don't publish a fixed UTC time for when the file drops. It regenerates every day, and rather than watching a clock, the simplest path is a webhook. Add one in your account under API keys, and we post a dataset.updated event the moment each daily refresh finishes, listing which datasets changed. Your job can then pull the fresh file from the download API automatically.
On older snapshots, the download always serves the current day's export rather than an archive of past days. So if you want to diff over time, the clean way is to pull the file each day when the webhook fires and keep your own copies to compare against. If that's central to what you're doing, happy to walk you through a simple setup.
Downloaded the JSONL and was honestly surprised how clean the WHOIS fields came out, barely any cleanup needed on my end. The daily refresh at a one-time price feels like a pretty solid deal for this kind of dataset.
@poyrazsucu89403 Hello Poyraz,
Thanks, that's great to hear.
Downloaded the JSONL sample and the WHOIS fields came through clean without the usual parse mess I get from other dumps. Pricing being one-time instead of subscription-based is honestly a breath of fresh air for anyone doing one-off research projects.