What is llms.txt and Why Does Your Website Need It?
When AI models (such as ChatGPT Search, Perplexity AI, Claude Search, and NotebookLM) search the web to answer user questions about your product, library, or company, they face heavy HTML boilerplate (scripts, navbars, ads, trackers) that dilutes context and wastes tokens.
`llms.txt` (specified at llmstxt.org) is the standardized markdown file hosted at `https://yourdomain.com/llms.txt` (similar to `robots.txt`) that provides AI search crawlers with a clean, curated markdown roadmap of your most important documentation and guides.