An llms txt file is a proposed standard: a short markdown file at yoursite.com/llms.txt that gives AI agents a curated map of your most useful pages. Google says Google Search ignores it, so it will neither help nor hurt your rankings. Other services may read it. Our verdict: it is cheap to add and worth an hour, but it is not a ranking lever.
What is llms.txt?
llms.txt is a plain markdown file that tells AI agents what your site is and which pages answer which questions. It is a guide, not a rulebook: it holds no crawl permissions and it does not replace your sitemap.
The llmstxt.org proposal describes it as a proposal to standardise on using an /llms.txt file to provide information to help agents use a website. It sets out a markdown file with brief background information, guidance, and links to detailed markdown files. The file usually sits at the site root, but the proposal allows it at any path, such as /docs/llms.txt, covering the pages under that path.
The proposal draws the contrast with sitemaps itself: while sitemaps list all pages for search engines, llms.txt offers a curated overview for LLMs. The word that matters is curated. A file that dumps every URL on your site misses the point of the format.
Three parts of the spec are worth knowing before you write one:
- An H1 with the name of the project or site. The proposal calls this the only required section.
- Sections of links, each with a short note saying what the linked page covers, so an agent can choose without opening everything.
- An “Optional” section which, by convention, holds secondary links an agent can skip when a shorter context is needed.
The proposal also says the format is human and LLM readable but precise enough for fixed processing methods such as parsers and regex. That precision is why the file is easy to check, which we come back to below.
What does Google say about llms txt files?
Google says Google Search does not use llms.txt, and that creating one will neither harm nor help your visibility or rankings in Google Search.
The wording sits in Google’s guide to optimizing for generative AI features: you don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search, including its generative AI capabilities, as Google Search itself doesn’t use them. The same guide adds that it’s completely fine to create and maintain llms.txt files for other services or systems that use them, and that doing so will neither harm nor help your site’s visibility or rankings in Google Search, as Google Search ignores them.
Google logged the change on its Search documentation updates page as “Clarifying guidance on llms.txt files”: a note added to the AI optimization guide clarifying Google Search’s usage of the files. The note was added in June 2026.
So for Google, including AI Overviews and AI Mode, the file is neutral. The same guide says those features are rooted in Google’s core Search ranking and quality systems, and that from Google Search’s perspective, optimizing for generative AI search is still SEO. If your goal is Google, your time goes on indexing, content and technical health, not on a text file.
Who reads llms.txt, if Google does not?
The honest answer: agents and tools that choose to, and the pages we cite do not name which AI assistants do. What the proposal does report is adoption on the publishing side.
According to the llmstxt.org page, thousands of sites publish an llms.txt file, documentation platforms generate one automatically, and Chrome’s Lighthouse audits sites for one as part of its agentic browsing checks. The author also writes that the expectation was that llms.txt would mainly be useful for inference rather than training, and that is how it has been used.
Read that carefully. Inference use means an agent looks at the file while answering a question or completing a task, then follows the links it needs. That is a real use case for documentation-heavy sites, software products and anyone whose visitors increasingly arrive through an assistant. It is not evidence that any particular chatbot will cite you because you published one. We never promise that, and you should be wary of anyone who does.
Does your site need an llms txt file?
Need, no. Benefit, possibly, at very low cost. Our recommendation is to add a short, accurate file once the work that actually moves visibility is done.
Here is how we weigh it against the jobs that come first:
| Task | Affects Google Search? | Affects AI agents? | Effort |
|---|---|---|---|
| Let AI crawlers through robots.txt and the firewall | Indirectly, via the same crawl health | Yes: a blocked crawler cannot read anything | Low to medium |
| Fix indexing and page templates (one H1, clean titles) | Yes | Yes, through the pages they read | Medium |
| Write pages that answer real questions directly | Yes | Yes | Ongoing |
| Publish an llms.txt file | No: Google Search ignores it | Possibly, for agents that read it | Low |
The order matters more than the file. If a firewall returns 403 to AI crawlers, a perfect llms.txt is a map nobody can follow. Before you write one, check what your site sends back to AI user agents with our free AI visibility checker. It reads your robots.txt rules for 11 AI crawlers and requests your homepage once with each crawler’s user agent. That is the checker’s own request, not the real crawler’s, and it covers the homepage only, so treat a pass as a good sign rather than proof.
Sites where we think the file earns its hour: software and API products with documentation, services businesses with a few key pages an assistant should find (pricing model, service areas, contact), and any site already being audited by tools that look for it. Sites where it can wait: anything with indexing problems, blocked crawlers or thin content.
How do you write and check an llms.txt file?
Write it by hand, then check it against the spec yourself. An llms.txt generator can give you a first draft, but the value of the file is the curation and the notes, which only you can write well. Our llms.txt example and template has a full file you can copy.
- Start with an H1 carrying your site or company name.
- Add a one or two line summary in a blockquote saying what you do and for whom.
- Group your most useful pages under H2 headings such as Services, Company and Support.
- Write each link as a markdown link followed by a colon and a short note on what the page answers.
- Put secondary pages (blog archive, older resources) under an Optional heading.
- Save it as llms.txt at your site root, served as plain text, and open it in a browser to confirm it loads.
- Review it whenever a key page moves or a service changes. A stale map is worse than none.
On generators: the llmstxt.org page notes that documentation platforms generate one automatically. That suits a docs site whose structure already is the map. For a business site, an auto-generated list of every URL is the sitemap again, which the proposal explicitly contrasts llms.txt against. Use the output as a draft and cut it down.
On validation: you do not need a special llms.txt validator to catch the mistakes we see. Because the format is precise enough for parsers and regex, a simple read-through against this checklist does the job:
- Exactly one H1, and it is the site name.
- Every link resolves and points to a live page, not a redirect chain or a 404.
- Every link has a note after it.
- No marketing copy, keyword lists or claims you could not defend on the page itself.
- The file returns 200 to a plain request, and your firewall does not block it for bot user agents.
What we have seen
On one SEO Autopilot engagement, a US healthcare analytics company on WordPress behind a firewall, the llms.txt component of our AI Visibility score went from 0 to 85 between the baseline frozen on 15 July 2026 and the re-measure on 18-19 August 2026, across all 39 sitemap URLs. Overall AI Visibility moved from 18.35 to 66.35.
Two caveats keep that honest. First, llms.txt is a 15% weight in our own AI Visibility rubric, a score we use to track readiness; it is not a Google ranking factor, and Google says plainly it ignores the file. Second, the larger problem on that site was access: its firewall returned 403 to OpenAI’s and Anthropic’s search crawlers on every page, because a generic bad-bot rule matched the substring “searchbot”. Crawler access, at a 20% weight, mattered more than the file. Our AI crawlers list covers the user agents to allow.
That is the pattern we expect to keep seeing: llms.txt is the easy, visible task, and the gains sit in the less visible work around it. It is one of the items we handle in SEO Autopilot, alongside crawler access, schema and content.
Frequently Asked Questions
Does llms.txt help SEO?
Not for Google Search. Google’s AI optimization guide says Google Search ignores llms.txt files, so creating one will neither harm nor help your visibility or rankings there. It may help agents and services that choose to read the file.
Is llms.txt the same as robots.txt?
No. robots.txt holds crawl rules for bots. llms.txt is a markdown guide to your most useful pages, with a short note on each link. It grants or blocks nothing, so crawler access still has to be fixed in robots.txt and at your firewall.
Where do I put my llms.txt file?
At the root of your site, so it loads at yoursite.com/llms.txt. The llmstxt.org proposal also allows the file at a subpath, such as /docs/llms.txt, covering the pages under that path.
Do I need an llms.txt generator?
No. A generator can produce a first draft, but the file is meant to be a curated overview rather than a list of every URL. In our experience, writing it by hand with a note on each link is about an hour’s work for a small business site.
How do I validate my llms.txt file?
Check that it has one H1 with your site name, that every link resolves and carries a note, that secondary links sit under an Optional heading, and that the file returns 200 to a plain request without being blocked by your firewall.