Do you know which AI bots are reading your site?

Most of us check Search Console every week. Far fewer people look at which AI crawlers are visiting: GPTBot, ClaudeBot, PerplexityBot, Google-Extended and the rest.

One thing that's shaped how I think about AI visibility: if an assistant's crawler never fetches your pricing or docs pages, it can't describe them accurately, however good the content is.

A few questions for anyone running a site:

• Do you allow or block AI crawlers in robots.txt, and why?

• Have you ever checked your logs for them?

• If you did, which pages did they actually fetch?

I'll reply to everyone with what we've seen.

Tanmay