Tell AI assistants what your site is actually about
robots.txt says what crawlers may not read. llms.txt says what is worth reading. Check whether yours is valid, or generate a starter file from your sitemap in one click.
Without llms.txt, an AI has to reverse-engineer your site from whatever it crawled
Language models do not browse the way people do. They arrive with a question, pull a handful of pages, and assemble an answer. If nothing tells them how your site is organised or which pages carry authority, they guess — and they guess from navigation labels and page titles.
Your hierarchy is invisible
A crawler sees a flat list of URLs. It cannot tell that your pricing page matters more than a 2023 changelog entry, or that six of your posts are one cluster. llms.txt makes that hierarchy explicit.
Your one-line pitch is missing
The summary blockquote under the title is the single highest-leverage line in the file. It is often what a model repeats when someone asks what your company does.
Low-value pages compete
Tag archives, pagination and thin utility pages get crawled alongside your best work. A curated file points models at what you would actually want quoted.
What a valid llms.txt contains
A single H1 with your site name
Exactly one # Heading at the top. More than one confuses parsers about which is canonical; zero leaves the file unattributed.
A summary blockquote
One > line directly under the title, describing what the site offers in a sentence. This is the part most generated files omit and the part that matters most.
Section headings
## Docs, ## Products, ## Guides — grouping that mirrors how a person would ask about your site, not how your CMS happens to store it.
Annotated markdown links
Each link followed by a short description of what the page covers. The annotation is what lets a model choose between two similar URLs without fetching both.
Validate your llms.txt, or generate one from your sitemap
Enter a domain. If an llms.txt exists we check it against the spec and show you a preview. If it does not, we build a starter file from your sitemap that you can edit and upload.
Reads only public files. Nothing is stored.
What the checks are testing for
Spec-complete
Title, summary, sections and annotated links all present. Assistants can parse your structure without inference.
Usable, incomplete
Typically a missing summary line or no section grouping. Functional, but you are leaving the most valuable signal on the table.
No file at all
Nothing curated for AI systems. Generate a starter file below, edit the summary, group the sections, and upload it to your site root.
llms.txt is a young convention, not a ranking factor
What it will not do
Adding llms.txt does not make you rank, does not guarantee citation, and is not confirmed as a signal by any major model provider. Anyone selling it as a ranking lever is overstating it.
Why it is still worth ten minutes
It costs almost nothing, it removes ambiguity about your positioning, and adoption is growing quickly among documentation sites and SaaS products. Cheap, reversible, and it forces you to articulate what your site is for — which is useful regardless.
llms.txt, answered plainly
Where does the file go?
At your site root, served as plain text: yourdomain.com/llms.txt. Same location convention as robots.txt.
Does it replace robots.txt?
No, they do opposite jobs. robots.txt restricts access; llms.txt curates and describes. You want both, and they should not contradict each other — check your crawler access with our AI crawler tool.
My CMS generates one automatically. Is that enough?
Usually not. Auto-generated files tend to dump every URL without grouping or annotation, and most omit the summary blockquote entirely. A generated file is a reasonable starting point that still needs a human pass.
How large should it be?
Curated, not exhaustive. A few dozen well-annotated links beats several hundred bare URLs. If your file is hundreds of kilobytes, it is a sitemap wearing a costume.
How often should I update it?
Whenever your site structure changes meaningfully. Watch for staleness — a file listing pages you deleted months ago actively misleads the systems reading it.
A good llms.txt is a symptom of a well-structured site, not a substitute for one
If writing the summary line is hard, that is usually a positioning problem rather than a file-format problem — and it is worth solving either way.