Hey PH! I'm Johan, and I built LLCrawler because I noticed a blind spot in the SEO world. Everyone optimizes for Google, but nobody is checking whether AI models — ChatGPT, Claude, Perplexity — can actually find and cite their website. These models are becoming a real traffic channel, and most sites are invisible to them. I kept asking myself: does my site have the right structured data? Is my robots.txt blocking GPTBot? Do I even have an llms.txt file? There was no tool to check all of this in one place. So I built one. **How it works:** 1. Paste your URL 2. LLCrawler crawls your site (just like an AI model would) 3. You get a score out of 100 with specific, actionable fixes It checks 6 categories: structured data, content structure, meta tags, AI readiness signals, sitemap, and performance. Each issue tells you exactly what to fix and why it matters for LLM visibility. Some things I've seen in the wild: - Sites blocking GPTBot in robots.txt without knowing it - Missing JSON-LD that makes sites invisible to AI citations - SPAs that render blank pages for AI crawlers (no SSR) - Sites with no llms.txt (the new standard for talking to AI crawlers) The tool is completely free, no signup required. I built it with Rails, and yes — LLCrawler scores 100/100 on its own analyzer (you can't sell gum if you don't chew it yourself). Would love your feedback. Try it on your own site and let me know what you think!

LLCrawler

Find out if ChatGPT can see your website

Find out if ChatGPT can see your website