llms.txt
Updated 2026-09-15
llms.txt is a proposed convention: a markdown file at a site's root that lists its most important pages and summarises what they contain, intended to help language models find authoritative content quickly. It is a community proposal rather than a standard any major engine has committed to honouring.
What it does today
In practice its value is internal as much as external: writing one forces a site to decide which pages are canonical for which questions, which is the same exercise that improves answer-first structure and internal linking. Whether a given engine reads the file is not something a site can verify from the outside, so it should be treated as cheap and speculative — a file to add, not a strategy to build on. Crawler permissions in robots.txt, by contrast, have measurable effects.
Illustrative file (example structure)
A short list: the entity page, the methodology page, the pricing page and four guides, each with one sentence saying what question it answers. Ten minutes of work, no risk — and the same list is a useful audit of whether those pages actually answer those questions.
Common mistakes
- Presenting llms.txt to stakeholders as a ranking mechanism. No engine has published a commitment to it.
- Maintaining a file that drifts out of date, which describes a site that no longer exists.
- Using it instead of robots.txt crawler decisions, which are the controls that verifiably matter.
Frequently asked questions
- Should we add llms.txt?
- If it takes an hour and stays current, there is little reason not to — but budget it as an experiment, and do not report it as a cause if visibility changes.
- Does it replace a sitemap?
- No. A sitemap is read by crawlers that verifiably use it; llms.txt is a summary aimed at models, with no guarantee of being read at all.
