Clean docs for LLMs
Crawl docs. Compose concise llms.txt.
5 followers
Crawl docs. Compose concise llms.txt.
5 followers
Crawl docs and compose optimized llm.txt files. Crawls any documentation site and produces clean, optimized text files perfect for LLM context windows. Fast, deterministic, and robots-aware.
Moonlighter
llms.txt Generator - Clean Docs for AI Assistants
Problem: Pasting documentation into Cursor/Claude gives you HTML soup that wastes context windows.
Solution: Generate clean, optimized text files from any docs site.
Why This Exists:
Raw HTML kills your token budget
Navigation/ads mixed with actual content
"Content too long" errors before you even ask questions
Manual extraction takes forever
What It Does:
Crawls docs sites. Outputs clean markdown files sized for LLM context windows.
Respects robots.txt and rate limits
Strips navigation, ads, boilerplate
Preserves code blocks and formatting
Two outputs: optimized `llm.txt` + complete `llms-full.txt`
Use Cases:
- Learning new APIs (Stripe, AWS, etc.)
- Feeding clean context to AI coding assistants
- Research/comparison across multiple services
- Team onboarding without documentation hell
How:
1. Paste docs URL
2. Wait 90 seconds
3. Download clean files
Moonlighter
Here are some popular docs for you to try:
1. https://learning.postman.com/docs
2. https://fastapi.tiangolo.com/
3. https://docs.python.org/3/library/heapq.html
4. https://docs.gitlab.com/