// guide

What is llms.txt?
844K sites have it. Here's what it actually does.

llms.txt is a markdown file that tells AI models how to understand and navigate your website. Anthropic, Cloudflare, and Stripe all have one. ChatGPT doesn't read it. Here's the complete, honest guide.

// quick answer

llms.txt is a plain markdown file at yourdomain.com/llms.txtthat provides AI models with a structured summary of your site — what it does, what pages exist, and how to navigate them. It's like a README for your website, written for LLMs. Implementation takes under 30 minutes and costs nothing to maintain.

What llms.txt actually is

llms.txt was proposed by Jeremy Howard (co-creator of fast.ai) in 2024. The idea is simple: as AI models increasingly browse the web to answer questions, HTML pages are noisy and token-expensive. A clean markdown summary at a known location gives models exactly what they need.

It sits alongside robots.txt and sitemap.xmlin your domain root. Unlike those files, it's written for AI comprehension — not crawler control or search indexing.

robots.txt

Controls what crawlers can access

All crawlers

sitemap.xml

Lists all pages for search indexing

Search engines

llms.txt

Summarises site content for AI models

LLM-powered tools

Which AI tools actually read llms.txt?

This is the most important question — and the answer is nuanced. Support is uneven across AI tools.

Claude (Anthropic)

Anthropic publishes its own llms.txt. Claude's web browsing feature may use it when exploring sites. Anthropic has not publicly confirmed the mechanism.

Likely yes

ChatGPT (OpenAI)

OpenAI has explicitly stated that ChatGPT's browsing does not use llms.txt. The file is ignored.

Gemini (Google)

No official confirmation. Google has extensive web crawling infrastructure. Gemini's web grounding may use llms.txt.

Possibly

Perplexity

Perplexity's AI search indexes structured web content. llms.txt aligns with its content consumption model.

Likely yes

Cursor / Claude Code

AI coding assistants that browse documentation use llms.txt to understand project structure without parsing full HTML.

Yes

Bottom line: llms.txt does not guarantee any AI will find or use your content. It is a best-effort standard with no enforcement mechanism. Its value is in improving accuracy when AI tools do read it — not in guaranteeing discovery.

Adoption: who has implemented llms.txt

844K+

Websites

By October 2025

2024

Proposed by

Jeremy Howard

Major adopters

Anthropic, Cloudflare, Stripe

Notable adopters include Anthropic (anthropic.com/llms.txt), Cloudflare (cloudflare.com/llms.txt), Stripe, and thousands of developer-tool and SaaS companies. The rapid adoption suggests the community views it as a low-cost, future-proof investment regardless of current AI support levels.

How to write a llms.txt file

The spec is intentionally minimal. A valid llms.txt needs five elements:

H1 headingRequired

Your site or product name. One H1 only.

Blockquote descriptionRequired

A one-sentence description of what the site does, in a > blockquote.

Section headings (H2)Optional

Organise links by section: Documentation, API, Guides, etc.

Markdown linksRequired

Links to your most important pages. Use relative URLs where possible.

Optional sectionOptional

An ## Optional section for supplementary links the AI may use if relevant.

Example llms.txt (supermd.dev)

# SuperMD

> LLM-optimised markdown generation for the AI era. Convert files, generate context files, and build AI-ready documentation.

## What SuperMD does

SuperMD generates every markdown file your AI consumes — optimised for fewer tokens, cleaner structure, and better model comprehension.

## Tools

- [markitdown](/markitdown): Convert PDF, DOCX, XLSX, and images to LLM-optimised markdown
- [claude.md](/claude-md): Generate CLAUDE.md from a GitHub URL
- [skill.md](/skill-md): Create Claude Code skill definitions
- [llms.txt](/llms-txt): Generate spec-compliant llms.txt for any website
- [context.md](/context-md): Build reusable context files for any topic
- [design.md](/design-md): Structured feature design documents

## Optional

- [Pricing](/pricing): Free tier and paid plans
- [What is Markdown](/what-is-markdown): Complete guide to .md files

Best practices for llms.txt

→

Keep it under 4,000 tokens

AI models have context limits. A concise llms.txt is more reliably consumed than a verbose one. For exhaustive documentation, use a companion llms-full.txt.

→

Use a companion llms-full.txt

Some tools support llms-full.txt for full documentation content. Keep llms.txt as the compact index; llms-full.txt as the complete reference.

→

Link to machine-readable content

Link to markdown or plain-text versions of your docs where available, not HTML pages. AI models process markdown with fewer tokens and less noise.

→

Update it when content changes

Stale llms.txt files cause AI models to reference outdated information. Automate updates when major pages are added or renamed.

→

Serve with correct Content-Type

Serve llms.txt as text/plain or text/markdown. Some AI crawlers check Content-Type before processing.

Frequently asked questions

What is llms.txt?

llms.txt is a markdown file placed at the root of a website (e.g. example.com/llms.txt) that provides AI models and LLM-powered crawlers with a structured summary of the site's content and purpose. It was proposed by Jeremy Howard (fast.ai) and is analogous to robots.txt — but instead of controlling crawler access, it gives AI models the context they need to accurately represent your site's content.

Does llms.txt actually work for SEO?

It depends on which AI model you're targeting. ChatGPT (OpenAI) has explicitly stated it does not use llms.txt. Claude and Gemini may use it during web browsing. Perplexity and other AI search tools likely index it. There is no confirmed ranking benefit from llms.txt alone — but it helps AI models accurately represent your site's content when they do crawl it, reducing hallucinations about your product or documentation.

What should be in a llms.txt file?

A llms.txt file should contain: (1) An H1 heading with the site/product name. (2) A brief description of what the site does. (3) Structured sections linking to key pages — documentation, API reference, guides. (4) Optional: a note on how to interpret the content. The file should be plain markdown, under 4,000 tokens, and updated when major content changes. A companion llms-full.txt can contain more verbose documentation.

Who created the llms.txt standard?

llms.txt was proposed by Jeremy Howard, co-creator of fast.ai and the ULMFiT language model pre-training approach. He published the initial specification in 2024. The standard is informal — there is no governing body — but it has been widely adopted, with over 844,000 websites implementing it by October 2025, including Anthropic, Cloudflare, and Stripe.

What is the difference between llms.txt and robots.txt?

robots.txt tells crawlers what they are and are not allowed to access. llms.txt tells AI models what your site is about and provides structured links to important content. robots.txt is machine-enforced and universally respected. llms.txt is informational and voluntary — AI models choose whether to read and use it. Both files live at the domain root.

Should I implement llms.txt on my website?

Yes, especially if your site has documentation, an API, or content that AI models should reference accurately. Implementation takes under 30 minutes, carries zero downside risk, and provides genuine value for AI tools that do read it. For SaaS products, developer tools, and documentation sites, it directly reduces hallucinations about your product in AI-generated answers.

Generate your llms.txt in seconds

Paste your URL. Get a spec-compliant llms.txt so any LLM can navigate your site without hallucinating your content.

What is llms.txt?844K sites have it. Here's what it actually does.