What is llms.txt? The complete 2026 guide

The 30-second answer

llms.txt is a plain-text file you place at the root of your website (like yourdomain.com/llms.txt) that tells AI assistants — ChatGPT, Claude, Perplexity, and others — which pages on your site are worth reading and how they should interpret them. Think of it as robots.txt and sitemap.xml had a baby, but for the AI era.

The proposal was introduced by Jeremy Howard in September 2024 at llmstxt.org and has since been adopted by companies including Anthropic, Cloudflare, Perplexity, Zapier, Shopify, and many others.

Why does llms.txt exist?

AI assistants have a fundamental problem when they visit your website: your homepage is designed for humans. It has navigation bars, cookie banners, marketing copy, and pretty pictures — and often relies on JavaScript to render meaningful content. That's fine for people. It's terrible for a language model trying to answer a customer's question in real time.

llms.txt solves this by giving the AI a hand-curated map of your site in the format it actually prefers: clean Markdown, with links and short descriptions of what each page covers. No CSS. No hydration mismatches. Just structured content.

What does a good llms.txt file look like?

The format is simple Markdown, with a specific structure:

# Your Site Name

> Short description of what your site is about.

Optional deeper context in a paragraph or two.

## Docs
- [Getting started](https://yoursite.com/docs/getting-started): Set up your first project.
- [API reference](https://yoursite.com/docs/api): Full API docs.

## Blog
- [Announcement post](https://yoursite.com/blog/launch): Why we built this.

## Company
- [About](https://yoursite.com/about): Who we are.
- [Pricing](https://yoursite.com/pricing): How it works.

The # heading is your site name. The > blockquote is a short one-line summary. Each ## heading is a section, and each bullet is a[Title](URL): description.

Do AI systems actually use it?

As of mid-2026: partially, and increasingly. Here's where things stand:

• Anthropic (Claude) and Perplexity read llms.txt when available.
• OpenAI (ChatGPT) hasn't officially confirmed usage but many sites report that adding llms.txt improved citation quality in ChatGPT answers.
• Google AI Overviews uses your regular sitemap and content — llms.txt isn't official for them yet, but Google-Extended (Gemini training) may consult it.
• Many second-tier AI crawlers (Kagi, Andi, You.com) explicitly reference llms.txt in their docs.

Even if a specific AI system doesn't read it today, adding llms.txt is nearly free — a few minutes of work — and gives you a machine-friendly summary of your site that will be used more as the standard matures.

How to add llms.txt to your site

1. Generate the file. You can hand-write it or use our free llms.txt Generator — it crawls your site, picks the most useful pages, and produces a properly formatted file in seconds.
2. Edit the descriptions. The auto-generated descriptions come from your meta tags and OG data. Rewrite them in your voice — this is what an AI will "see" when it summarizes your site.
3. Upload it to your site root. On WordPress: via File Manager to public_html. On Vercel/Netlify: drop it in /public. On Shopify or Squarespace: this is trickier — see the platform-specific instructions in our tool.
4. Verify it works. Visit yourdomain.com/llms.txt in a browser. You should see plain text, not an HTML page.
5. Add a companion robots.txt entry. Add Allow: /llms.txt under User-agent: * in your robots.txt so crawlers know it exists.

Common mistakes to avoid

• Skipping descriptions. An llms.txt with just links is missing the point — the descriptions are what AI uses to decide which pages are relevant to a query.
• Including too many pages. This isn't your sitemap. Include the 10-30 pages that best represent what your site offers. Everything else is noise.
• Listing broken or gated URLs. If an AI can't actually fetch the page, listing it in llms.txt hurts more than helps.
• Forgetting to update it. Regenerate llms.txt when you launch major new sections or restructure your site.

llms.txt vs llms-full.txt

There's a secondary standard called llms-full.txt that includes the full Markdown content of every page inline. It's useful for documentation-heavy sites where an AI might want to answer questions without fetching individual pages. Most sites should start with just llms.txt — it's simpler and covers the 80% case.

Bottom line

llms.txt is a low-effort, high-signal addition to your site's AI-readiness stack. It won't fix a site that AI crawlers can't reach at all (that's what our free readiness check is for), but for sites that are already crawlable, it meaningfully improves how AI assistants describe and cite you.

Generate yours in about 30 seconds using our free llms.txt Generator. It includes a companion robots.txt snippet and platform-specific install instructions for WordPress, Shopify, Squarespace, Wix, Webflow, Vercel, Netlify, and Cloudflare Pages.