What Is llms.txt? What It Does, What It Doesn’t, And Whether You Need One In 2026

Short answer: llms.txt is a plain Markdown file placed at the root of a website (yoursite.com/llms.txt) that gives large language models a short, curated map of the site’s most useful pages. It was proposed in September 2024 as a way to help AI tools read a site at the moment they answer a question. It is a community proposal, not an official standard, and Google has said you do not need it to appear in AI Overviews or AI Mode. It can help AI agents and coding assistants navigate documentation, but there is no evidence that it makes ChatGPT, Perplexity or Google cite you more often.

That gap between the hype and the evidence is why so many teams are confused. Plenty of agencies sell llms.txt setup as a GEO tactic, while Google’s own search team compares it to the old keywords meta tag. This guide explains what the file actually is, how to write one properly, what the major platforms have said about it, and how to decide whether it belongs on your site.

What Is llms.txt?

llms.txt is a proposal for a single text file, written in Markdown, that tells language models which pages on a site matter most and what each one contains. Jeremy Howard, co-founder of Answer.AI, published the idea on September 3, 2024, along with a specification at llmstxt.org.

The problem it set out to solve is practical. Web pages are full of navigation, scripts, cookie banners and ads, and a model working inside a limited context window wastes space reading all of it. A short Markdown index lets a tool skip the clutter and go straight to clean descriptions and links. The proposal says the file is mainly meant to be used at inference time, when a model is answering a user, rather than as training data.

Two details are easy to miss:

• It is not access control. llms.txt does not block or allow anything. Controlling which bots can crawl your site is still the job of robots.txt.
• It is not a sitemap. A sitemap lists every URL you want indexed. llms.txt is meant to be a curated shortlist with context, closer to a table of contents written for a machine.

How Is llms.txt Different From robots.txt And sitemap.xml?

File	What it does	Who reads it	Is it a standard?
robots.txt	Tells crawlers which paths they may or may not fetch	Search engines and most AI crawlers	Yes, widely honored
sitemap.xml	Lists URLs you want discovered and indexed	Search engines	Yes
llms.txt	Offers a curated Markdown index of key pages with short descriptions	Some AI tools, agents and coding assistants	No, a community proposal
llms-full.txt	Bundles the full text of key pages into one Markdown file	Mostly documentation tools and agents	No, an extension of the proposal

If you only remember one row, remember the first. Blocking or allowing AI crawlers happens in robots.txt, not in llms.txt. Our guide to how to rank in ChatGPT covers which crawler tokens matter for visibility.

What Does An llms.txt File Look Like?

The specification uses a fixed Markdown shape so the file can be read by people and parsed by software:

1. An H1 heading with the name of the site or project. This is the only required element.
2. A blockquote with a one or two sentence summary of what the site is.
3. Optional paragraphs or bullet points with extra context.
4. One or more H2 sections, each containing a list of links written as [Page name](URL): short description.
5. An optional section titled “Optional” for links a tool can skip if it is short on space.

Here is a simple example for a fictional B2B software company:

# Northwind Analytics

> Northwind Analytics is a product analytics platform for B2B SaaS teams. It tracks feature adoption, retention and revenue by account.

Northwind is used by product, growth and customer success teams. Pricing is per tracked account.

## Product
- [Product overview](https://northwind.example/product): What the platform does and the main modules
- [Pricing](https://northwind.example/pricing): Plans, limits and what each tier includes
- [Integrations](https://northwind.example/integrations): Supported data sources and destinations

## Documentation
- [Quickstart](https://northwind.example/docs/quickstart): Install the tracker and send the first event
- [API reference](https://northwind.example/docs/api): Endpoints, authentication and rate limits

## Optional
- [Company blog](https://northwind.example/blog): Product updates and research

What Is llms-full.txt?

llms-full.txt is a companion file that goes a step further. Instead of linking to pages, it contains the full Markdown text of those pages in one long document, so an agent can load everything in a single request. The pattern grew out of documentation platforms, Mintlify in particular, and Anthropic’s own developer docs publish both a slim llms.txt index and a much larger full export. For a marketing site with 30 pages, llms-full.txt is rarely worth maintaining. For a large developer documentation set, it can be genuinely useful.

Who Actually Reads llms.txt?

This is the question that matters, and the honest answer is: fewer systems than the marketing around it suggests.

What Google Says

Google has been unusually direct. Its Search Central page on AI features and your website says there are no additional requirements to appear in AI Overviews or AI Mode, and that you do not need to create new machine readable files, AI text files or special markup to appear in those features.

Google’s John Mueller has gone further. He compared llms.txt to the keywords meta tag, a self-declared signal that search engines stopped trusting long ago because site owners could write anything in it. In a June 2026 episode of Google’s Search Off the Record podcast, reported by Search Engine Journal, he argued that a self-reported file cannot help an AI system choose one website over another. He did leave room for one use: once an automated agent is already on your site, a file like this might help it find its way around.

What The Data Says

Independent studies have not found a visibility effect. Search Engine Journal reported on an SE Ranking analysis of roughly 300,000 domains that found no link between having an llms.txt file and how often a domain was cited in AI answers.

Adoption is also lower than the buzz implies. Rankability’s monthly tracker found that 8.7% of the top 1,000 websites published an llms.txt file as of June 2026, with adoption concentrated in technology and developer-focused sites.

Where It Does Get Used

llms.txt is most at home in developer documentation. Coding assistants, IDE agents and documentation tools can be pointed at a project’s llms.txt or llms-full.txt to pull in clean, current reference material. That is a real use case, and it is the one the original proposal was built around. It is just not the same thing as ranking in AI search.

Does llms.txt Help With GEO Or AI Search Visibility?

Not in any way that has been demonstrated. AI search tools decide what to cite based on what they retrieve from their search indexes and how well a page answers the question. Our explainer on how language models decide who to cite walks through that process. A file that lists your favorite pages does not change how relevant, trusted or well linked those pages are.

The things that do move AI visibility are the same ones covered in our GEO vs SEO guide: being crawlable by the right bots, ranking for the sub-questions behind a prompt, writing passages that can be quoted on their own, and earning mentions on the third-party sites that AI answers lean on.

If an agency pitches llms.txt as the centerpiece of a generative engine optimization program, treat that as a warning sign. The best GEO agencies lead with research, content and digital PR, not with a text file.

When Is llms.txt Worth Adding?

Because it takes little effort and does no harm, adding one is reasonable in a few situations:

• You publish developer documentation or an API. Agents and coding assistants are the clearest beneficiaries.
• You expect AI agents to complete tasks on your site. Booking, buying or configuring something are the kinds of journeys Mueller suggested a file could support once an agent has arrived.
• Your platform generates it automatically. Many documentation tools and some CMS plugins now create the file for you, so the cost is close to zero.
• A client or stakeholder insists. Add it, but set expectations that it is a convenience for agents, not a ranking lever.

It is probably not worth your time if you run a small marketing or local business site and have not yet fixed the basics: crawl access for search bots, clear service pages, reviews and third-party coverage.

How To Create An llms.txt File

If you decide to add one, keep it short and accurate:

1. Pick 10 to 30 pages that matter most. Product or service pages, pricing, key documentation and a few cornerstone guides. Resist the urge to list everything.
2. Write one honest line per link. Describe what the page contains, not why you are the best. Promotional claims are exactly what makes self-reported files untrustworthy.
3. Follow the format. One H1, a blockquote summary, H2 sections with Markdown link lists, and an optional section last.
4. Serve it as plain text at the root. It should load at https://yourdomain.com/llms.txt without a login, redirect chain or JavaScript.
5. Make sure the linked pages are reachable. Check that robots.txt, your CDN and any bot protection are not blocking the AI crawlers you want, or the file points to pages they cannot read.
6. Review it whenever your site structure changes. A stale file with broken links is worse than none.

On WordPress, several SEO plugins can now generate the file. If yours does, review the output by hand before relying on it, since auto-generated versions often list too many low-value URLs.

Common Mistakes

• Treating it as a sitemap. Dumping hundreds of URLs defeats the purpose of a curated index.
• Using it to block bots. Disallow rules belong in robots.txt.
• Stuffing it with marketing copy. Systems that read the file still check the pages themselves.
• Linking to pages that bots cannot fetch. Gated, script-rendered or WAF-blocked pages are invisible no matter what the file says.
• Measuring success by the file alone. Track AI visibility with prompt testing and tools instead. Our list of AI visibility tracking tools is a good place to start.

Frequently Asked Questions

Is llms.txt an official standard?

No. It is a community proposal published by Jeremy Howard of Answer.AI in 2024. It has not been adopted by a standards body, and the major AI search providers have not committed to using it for ranking or citation.

Does Google use llms.txt?

Google says you do not need any new machine readable or AI text files to appear in AI Overviews or AI Mode, and its search team has compared llms.txt to the keywords meta tag.

Will llms.txt get me cited by ChatGPT or Perplexity?

There is no evidence that it does. An SE Ranking study of about 300,000 domains found no relationship between llms.txt adoption and AI citation frequency.

Can llms.txt hurt my site?

Not directly. The main risks are wasted time and a false sense that AI visibility has been handled.

What is the difference between llms.txt and llms-full.txt?

llms.txt is a short index of links with descriptions. llms-full.txt contains the full text of the linked pages in a single Markdown file, which is mostly useful for documentation sites and AI agents.

The Bottom Line

llms.txt is a tidy idea for helping AI tools read a website, and it has found a real home in developer documentation. As an AI search visibility tactic, the evidence is not there, and Google has said plainly that it is not needed. Add one if you have docs or expect agents to use your site, keep it short and honest, and put most of your effort into the work that actually earns citations. For a structured way to find out where you stand, see our guide to running an AI visibility audit.