Launched this week
Known Agents is "Google Analytics for bots". AI agents, crawlers, and scrapers are now half of your website's traffic, and they're becoming more important to your business every day. They're silently buying your products, reading your docs, and researching your company. Known Agents gives you realtime visibility into all of this activity. You can see which bots are visiting your site, the pages they’re most interested in, where they’re coming from, and which humans they're referring to you.
Known Agents
Hi everyone 👋
Almost half of your website's traffic isn't human anymore. It's bots and AI agents. (source)
Bots aren't new, but AI just made their activity a lot more meaningful. AI agents are now doing things like:
🛍️ Buying your products and booking reservations
💻 Reading your docs to implement your tools and code
📖 Scraping your content to train models
🔬 Researching your business for competitors
The problem is that they're completely hidden from you right now. You're blind to half of your visitors if you're only using a human analytics platform, and you can't optimize what you can't see.
Known Agents is "Google Analytics for bots". It lets you see exactly what they're doing on your pages, and how they're referring real humans from AI chat and search platforms. This is a new type of user that's only going to get more important.
It's free for the vast majority of websites, and takes 5 minutes to set up.
Check it out: https://knownagents.com
Let me know what you see in your realtime feed. Which bots are visiting, and what surprised you?
Feedback is appreciated!
@ghking Congrats 🙌
Once people see all these AI bots in their feed, what are your early users actually doing with the info? Blocking them, optimising for them, or just watching? Curious where this goes 👀
Known Agents
@francesco2689 thank you! Right now, it’s a mix of all three.
Every website now needs basic visibility into AI agents and LLM referrals, which are becoming a new category of traffic source and distribution channel, similar to search or social.
What you do with that information depends on your industry. For example, publishers and content creators can use these insights to control access with robots.txt and firewall rules. E-commerce companies can see where shopping agents are getting stuck before completing a purchase, then optimize their UI and checkout flows to improve conversion.
Very cool, a lot has been said lately about how much general internet traffic is bots, but I've never really been able to quantify exactly how that's affecting the sites I build. Being able to separate out agent traffic from scraping is huge too!
Known Agents
@david_pimentel Totally, there's so much more diversity within the bot ecosystem that people don't realize.
We're tracking around ~3,000 bots across 17 categories (e.g. AI Assistants, AI Coding Agents, AI Data Scrapers, Search Engine Crawlers, SEO Crawlers, etc.)
They all behave in different ways, and are interested in different kinds of content. It's really interesting to see the patterns.
I was watching a video yesterday about how the 'users' of the next great software products will be agents. I feel like this would allow people to better design and optimize their tools for bots. How could this help build for agents?
Known Agents
@tyler_farley1 Exactly, basically optimizing "Agent Experience" (AX) rather than just "User Experience" (UX).
Known Agents does exactly that. For example, you can replay individual agent sessions to see exactly what path they took navigating trough your website, including where they dropped off. You can use that information to adjust that page to be more agent-friendly (e.g. maybe there are hidden elements, problematic asynchronous loads, or unclear copy that need to be adjusted).
Great idea in a long overdue category. The LLM referral tracking alone is worth it - being able to see when ChatGPT or Claude is sending traffic to Boraspeak's website has reshaped how we think about SEO/AEO and content. Congrats @ghking!
Known Agents
@pdbthefifth Thanks!
Agreed, measuring and optimizing your content for LLM referrals from AI chat platforms (GEO, AEO, AIO) is becoming just as important as social and search (SEO) these days.
It's crazy how many people and businesses aren't looking into it at all, or don't even know that they should.
CatFacts
This is truly such a great idea. The internet is shifting away from humans. We’re already seeing more and more purchases on our site come from agents but it’s impossible to tell where they’re coming from. I’ll definitely be trying Known Agents out.
Known Agents
@kylevenn Thanks Kyle! Agreed, we see an extremely clear trend in the data. More and more traffic is shifting away from humans and toward agents every day, and it looks like the ratio will finally flip in the next few months. Agents will be the primary user of the internet (the "agentic web").