What Is llms.txt? How to Create One and Whether It Actually Works

Short answer: llms.txt is a plain Markdown file at the root of your website that lists your most important pages, each with a short note, so that language models and AI agents can find them quickly. It is cheap to create, but a benefit for rankings or AI citations is not proven: Google's documentation says you do not need AI text files to appear in its AI features, reports say Google does not support the file, and a Google representative has said that, as far as he knows, no AI system uses it. Add it after the basics (robots.txt, sitemap, readable HTML), never instead of them.

What is llms.txt?

llms.txt is a proposal, not an official web standard. It was published in 2024 by Jeremy Howard at llmstxt.org. The idea: sitemaps list all pages for search engines, but a language model has a limited context window and cannot digest a whole website. A short, curated file in Markdown can point it to the pages that matter.

According to the current specification, a file named llms.txt can sit at the root (/llms.txt) or at a subpath such as /docs/llms.txt. It contains these parts, in this order:

1. An H1 with the name of the site or project. This is the only required part.
2. A blockquote with a short summary.
3. Optional paragraphs or lists with more detail.
4. Optional sections introduced by H2 headings, each containing a list of links in the form [name](url), optionally followed by a colon and a note.

By convention, a section called Optional holds secondary links that an agent can skip when it needs a shorter context.

Here is a shortened version of our own file at seogrinder.com/llms.txt:

# SEOgrinder

> Free SEO analysis for public websites. Enter a web address and get a full report: speed, on-page SEO, accessibility, keywords, plus a prioritised list of what to fix first. No account needed.

## Tool

- [SEOgrinder](https://seogrinder.com/): the analysis tool
- [SEOgrinder as Markdown](https://seogrinder.com/index.md): what the tool does, as plain text

## Optional

- [Imprint](https://seogrinder.com/imprint): who runs the service

llms.txt, robots.txt and sitemap.xml: three different jobs

The three do not replace each other. A crawler that is blocked in robots.txt cannot read your llms.txt either, and a page that is empty without JavaScript is not rescued by a nice list of links.

Does llms.txt work? What we actually know

The honest answer is: nobody has shown a benefit, and according to reports the biggest search engine does not support the file. Here is what has been reported:

• Google does not support it. A report from a Google Search Central Live event says Gary Illyes stated that Google does not support llms.txt and does not plan to. The report comes from a photo posted on LinkedIn, not from a transcript.
• No AI system is known to use it. John Mueller of Google said that, as far as he knows, no AI system currently uses llms.txt, and compared it to the old keywords meta tag (Search Engine Journal).
• Google's own sites are no evidence. In December 2025 llms.txt files showed up on Google's developer sites. Mueller named a change in the CMS platform as the reason, and in January 2026 he answered the question whether this was an endorsement with a plain "no".
• One log file is an anecdote. In July 2025 a site owner posted a log file showing that OpenAI's crawler requested his llms.txt about every 15 minutes. That shows a request, not that the content is used in answers.

Sources: Search Engine Roundtable on Google and llms.txt and on OpenAI requests and on the Google developer sites. These are second-hand reports of statements on social media, so read the wording with care. Things can also change: check the current statements before you decide.

Our verdict: llms.txt is a low-cost, unproven extra. SEOgrinder files it under "Extras" in the AI readiness report, and while an extra is still open, the tool says so openly: these are suggestions from individual vendors, and an effect on Google is not proven.

Who should create one anyway?

The proposal grew out of a practical case: a coding agent fetches a library's documentation to get an API call right. For a documentation site with many pages, a clean index in Markdown is useful for exactly that. It is also useful for any site where you want agents to find the five pages that matter instead of guessing. Nothing is lost if it turns out not to matter, because writing the file takes minutes.

If you run a small local business site, the time is better spent on the basics: pages that show up in the HTML without JavaScript, correct opening hours and prices as text, and a robots.txt that lets the search bots in.

How to create an llms.txt in five minutes

1. Name the site in an H1. # Your Company is enough.
2. Write the summary in a blockquote. One or two plain sentences: what you offer, for whom, where. No slogans.
3. List your key pages under H2 headings. Use headings such as Products, Documentation or Contact.
4. Write each link as - [Name](https://example.com/page): what it contains. Use absolute addresses, and keep the notes short and concrete.
5. Put secondary pages under ## Optional. Legal pages and archives belong here.
6. Save the file as UTF-8 text and publish it at /llms.txt. It has to answer with status 200.
7. Keep it current. When a page moves, change the link. A list of dead links is worse than none.

You can also let your coding assistant do it. In SEOgrinder, open the AI readiness tab, find the llms.txt line under "Checks for AI agents" and press Copy agent prompt. If the file is missing, the prompt describes what to create and how to check it. If the file exists, you can expand the line to see what was found, and the prompt then asks the assistant to review it and change only what is wrong.

Should you add noindex to llms.txt?

Mueller said that it could make sense: sites might link to the file, and it could otherwise become indexed, which would be odd for users. He said so in a side remark, not as a rule, so weigh it yourself. For a text file you can do it with the response header X-Robots-Tag: noindex. The file does not duplicate a normal page, so there is no duplicate-content problem.

Common mistakes

• Pasting the whole sitemap into it. The point is a selection. Twenty well-described links beat two thousand bare ones.
• Marketing text in the summary. Models need facts: what, for whom, where.
• Relative or broken links. Use full addresses and check them.
• Blocking the file. A firewall rule or a robots.txt line that keeps bots away from /llms.txt defeats the purpose.
• Treating it as a ranking lever. Nothing in the sources above supports that.
• Forgetting it exists. A file that lists last year's pages misleads agents.

FAQ

Is llms.txt a ranking factor?

There is no evidence that it is. Google's documentation says AI text files are not needed for its AI features, and reports say Google does not support the file.

Will ChatGPT or Claude use my llms.txt?

Nobody outside the providers can say. Some AI companies publish an llms.txt for their own documentation, and a log file shows that at least one crawler requests such files. That does not show how, or whether, the content is used in answers.

Is llms.txt the same as robots.txt?

No. robots.txt controls access for crawlers. llms.txt describes content. See the table above.

Do I need a developer to add it?

No. It is a text file. If you use a CMS or a platform with its own root folder, you may need a plugin or a rewrite rule to serve it at /llms.txt.

How do I check that mine works?

Open https://yoursite.com/llms.txt in a browser. It should show the text, and the status should be 200. Then run the free scan: SEOgrinder reports whether the file exists and shows what it found.

Your next step

Check the basics first: run the free scan on your site and open the AI readiness tab. If robots.txt, sitemap and rules for AI crawlers are in order, create your llms.txt in five minutes and rescan.