What is llms.txt, and should you create one?

llms.txt is a plain Markdown file published at the root of a website (for example example.com/llms.txt) that gives language models a short summary of the site and a curated list of links to its most useful pages. It was proposed by Jeremy Howard of Answer.AI on 3 September 2024. It is cheap to create and harmless to publish, but it is not a search ranking factor: Google says its AI features do not need it. It is most useful for documentation, developer tools and other sites that AI assistants and coding agents read directly.

What problem llms.txt tries to solve

Websites are built for people and browsers. A page mixes the content a model needs with navigation, scripts, cookie banners and layout, and a whole site is usually far too large to fit in a model's context window. The llms.txt proposal gives site owners one small, predictable file where they can say, in plain language, what the site is and which pages matter most, so a model or an agent can fetch just those.

The proposal also suggests that sites offer a clean Markdown version of useful pages at the same URL with .md appended. That part is optional and separate from the llms.txt file itself.

The llms.txt format

The file is ordinary Markdown with a fixed order of sections. Only the first is required:

1. An H1 with the name of the site or project. This is the only required element.
2. A blockquote summary with the key facts someone needs to understand the rest of the file.
3. Optional free text (paragraphs or lists, but no headings) with further context, such as how to interpret the docs.
4. H2 sections, each containing a list of links in the form - [Title](url): notes, where the notes are optional.
5. An optional section named Optional, whose links can be skipped when a shorter context is needed.

A small, complete example:

# Example Docs

> Example is a scheduling API for clinics. These docs cover setup, the REST API and billing.

Prices on these pages are in US dollars. The v1 API is deprecated; use v2.

## Docs

- [Quick start](https://example.com/docs/quick-start/): Create an account and make a first API call
- [API reference](https://example.com/docs/api/): Every v2 endpoint with request and response examples

## Policies

- [Pricing](https://example.com/pricing/): Plans, limits and overage charges

## Optional

- [Changelog](https://example.com/changelog/): Release notes since 2024

Some documentation sites also publish an llms-full.txt file containing the full text of their docs in one file. That is a common convention rather than part of the core proposal.

Who actually reads llms.txt

This is where expectations need care. Google has been direct about its position: Gary Illyes said in July 2025 that Google does not support llms.txt and is not planning to, and Google's May 2026 guide to optimizing for its generative AI features says llms.txt is not needed for AI Overviews or AI Mode and is treated like any other text file. The same guide says Google's AI features draw on its normal search index and ranking systems, so standard SEO work is what counts there.

Outside Google, the clearest use today is with AI coding assistants and agents. A developer can point an assistant at a library's llms.txt, or a tool can fetch it on demand, to get a compact map of the documentation instead of crawling the whole site. That is why the file is most common on developer documentation sites. We are not aware of a major AI search engine that has publicly committed to using llms.txt when choosing which pages to cite, so treat any claim that it improves AI citations with caution.

Should you create one?

• Yes, if you publish documentation, an API or a developer product. Your users' AI assistants are likely to read your docs, and a curated map helps them find the right pages.
• Probably yes, if it takes an hour. For most sites the cost is small, and a clear summary and page list does no harm.
• Do not expect ranking or traffic gains. It will not change how Google ranks you or whether AI Overviews cite you. Crawlable pages, clear answers and a solid reputation matter far more; see how to get cited by ChatGPT and Perplexity.

How to create an llms.txt file

1. Pick the pages that answer real questions: product overview, documentation, pricing, policies, key guides. This is a curated list, not a sitemap.
2. Write a one or two sentence summary that states what you are and who you serve, in plain language.
3. Group the links into sections and add a short note to each link saying what the page answers.
4. Use absolute URLs, so the links work wherever the file is read.
5. Publish it at the root as /llms.txt, served as plain text, and make sure robots.txt does not block it.
6. Keep it current. Review it when you add or retire important pages.

Our free llms.txt generator builds a valid file in the browser. You can paste your sitemap.xml to import URLs grouped by folder, then trim the list and add notes. If you want to offer Markdown versions of pages as well, our HTML to Markdown converter can produce them.

How we approach this

We add llms.txt where a client has documentation or a product that people use through AI assistants, and we keep it short and maintained. For marketing sites we treat it as optional housekeeping and spend the effort on the things that do move AI visibility: content that answers real questions clearly, pages that are fast and crawlable, and accurate structured data.

Sources: llmstxt.org, Answer.AI's original proposal, Google's guide to optimizing for generative AI features, Search Engine Land on Google's position.