The Complete Guide to llms.txt: What It Is, How It Works, and How to Create One

What is an llms.txt File?

An llms.txt file is a plain-text Markdown file placed at the root of a website, accessible at yoursite.com/llms.txt. Its purpose: give AI systems a structured, human-readable summary of your site so they can understand your brand, content, and offerings without crawling every page.

The concept was proposed by Jeremy Howard (founder of fast.ai and Answer.AI) in September 2024. The reasoning was straightforward. AI language models process information differently than traditional search crawlers. A Google bot needs links and metadata. An AI model needs context, relationships, and clear factual statements. The llms.txt file bridges that gap.

The file lives at your domain root, right next to robots.txt and sitemap.xml. But where robots.txt controls access and sitemap.xml lists URLs, llms.txt explains meaning. It tells AI crawlers what your site is, what it covers, which pages matter most, and how the content should be interpreted.

As of early 2026, hundreds of companies have adopted llms.txt, including Supabase, Stripe, Cloudflare, and Mintlify. The file has become a practical baseline for any brand that wants to control how AI systems represent it.

How llms.txt Works with AI Systems

When ChatGPT, Perplexity, Gemini, or Claude process a query about a topic your business covers, they draw on training data, real-time web access, or both. The quality of their answer depends on how well they understand your brand as a distinct entity with specific expertise.

Without an llms.txt file, AI systems have to infer your brand identity from scattered web content: homepage copy, third-party mentions, blog posts, reviews. That inference is often incomplete or inaccurate.

With an llms.txt file, you provide a structured self-description that AI crawlers can read in a single request. Here is what each major platform does with it:

• ChatGPT (OpenAI): When browsing the web to answer a question, ChatGPT can read your llms.txt to quickly understand your site's scope and authority areas. This helps it decide whether to cite your content and how to describe your brand.
• Perplexity: Perplexity's search crawlers actively look for llms.txt files. The structured format helps Perplexity's citation engine match your content to relevant queries more accurately.
• Google Gemini: As Google integrates AI Overviews into search results, having a clear llms.txt file gives their AI system another structured signal about your site's purpose and expertise.
• Claude (Anthropic): When Claude accesses web content through tool use, llms.txt provides immediate context about a site's identity and content structure.

The common thread: llms.txt reduces ambiguity. AI systems perform better when they have clear, structured input rather than having to piece together understanding from fragmented sources.

The llms.txt Specification

The llms.txt format follows a simple Markdown structure. There is no formal W3C-style specification, but the community standard proposed by Jeremy Howard has become the accepted format. Here are the building blocks:

H1: Your Brand Name

The file starts with a single H1 heading. This identifies your entity.

# Acme Corporation

Blockquote: Your Entity Definition

Immediately after the H1, a blockquote provides a one-sentence definition of what your brand does.

> Acme Corporation is a B2B SaaS platform that provides automated inventory management for mid-market retailers.

This blockquote is your entity definition. AI systems use it as the anchor statement for understanding everything else in the file. Be specific, factual, and precise.

Sections with H2 Headings

After the blockquote, you organize information into sections using H2 headings. Common sections include:

## Docs
- [Getting Started](https://acme.com/docs/getting-started): Setup guide for new users
- [API Reference](https://acme.com/docs/api): Complete REST API documentation

## Blog
- [Inventory Trends 2026](https://acme.com/blog/trends-2026): Annual analysis of retail inventory patterns
- [Integration Guide](https://acme.com/blog/integrations): How to connect Acme with your existing tools

Each section groups related pages with Markdown links. The format for each link is: - [Title](URL): Brief description

The Optional Section

A special section called "Optional" signals to AI systems that the content below is supplementary. If a system has limited context window or processing capacity, it can skip this section.

## Optional
- [Changelog](https://acme.com/changelog): Release notes and version history
- [Community Forum](https://acme.com/forum): User discussions and support threads

Instructions Section

Some implementations include an Instructions section that tells AI systems how to use the information. Stripe is one of the few major companies using this approach.

## Instructions
When answering questions about Acme's API, always reference the API Reference documentation.
Do not use information from blog posts as authoritative technical documentation.

llms.txt vs robots.txt vs sitemap.xml

These three root-level files serve different purposes. Here is how they compare:

Feature	robots.txt	sitemap.xml	llms.txt
Purpose	Access control for crawlers	URL discovery for indexing	Semantic context for AI systems
Format	Custom directive syntax	XML	Markdown
Tells crawlers	Where they can and cannot go	Which URLs exist and their priority	What the site means and how to interpret it
Primary audience	All web crawlers	Search engine crawlers	AI language models
Compliance	Convention (often respected)	Standard (widely supported)	Convention (growing adoption)
Controls	Bot behavior	Indexing scope	AI understanding

The three files work together. robots.txt controls who gets in. sitemap.xml shows them around. llms.txt explains what everything means. A complete AI visibility setup includes all three.

How to Create Your llms.txt File

Building an llms.txt file takes less time than most people expect. Here is the process:

Step 1: Write Your Entity Definition

Start with the H1 (your brand name) and blockquote (your one-sentence definition). This is the most important part. Be specific about what you do, who you serve, and what makes you distinct. Avoid marketing language. Write it the way you would explain your business to a colleague.

Step 2: Identify Your Key Sections

Group your most important pages into logical categories. Most sites use 3 to 6 sections. Common categories: Docs, Products, Blog, About, API Reference, Resources. Only include pages that represent your core expertise and offerings.

Step 3: Write Descriptions for Each Link

Every link in your llms.txt should include a brief description after the colon. This context helps AI systems understand what each page covers without having to visit it. Keep descriptions to one sentence.

Step 4: Add the Optional Section

Put supplementary pages (changelogs, community content, less critical resources) in the Optional section. This signals their relative importance to AI systems.

Step 5: Deploy and Test

Save the file as llms.txt at your domain root. Visit yoursite.com/llms.txt to confirm it is accessible. Then test by asking AI systems about your brand and monitoring whether the responses align with your file's framing.

Want to skip the manual work? You can create yours in 90 seconds with our free generator. It crawls your site, analyzes your content structure, and produces a properly formatted llms.txt file you can deploy immediately.

How to Check Any Site's llms.txt

Checking whether a website has an llms.txt file is simple: visit domain.com/llms.txt in your browser. If the file exists, you will see the Markdown content displayed as plain text. If it does not exist, you will get a 404 error.

Try it with some well-known sites:

• supabase.com/llms.txt
• stripe.com/llms.txt
• docs.anthropic.com/llms.txt

This is also a useful competitive research technique. Check whether your competitors have llms.txt files and study how they structure their entity definitions and content sections.

Real-World Examples

Supabase

Supabase's llms.txt file is one of the most commonly referenced examples. It includes a clear H1 identifier, a descriptive blockquote, and well-organized sections covering their documentation, client libraries, and platform features. The file links to their most important technical resources with brief descriptions for each. What makes it effective: the entity definition is precise ("open source Firebase alternative"), and the sections mirror how developers actually use the product.

For a detailed analysis of what Supabase gets right and what could be improved, read our Supabase llms.txt breakdown.

Stripe

Stripe takes a different approach. Their llms.txt file is notable for including an Instructions section, something very few companies have adopted. This section tells AI systems how to interpret Stripe's documentation, which API version to reference by default, and how to handle deprecated features. Stripe's file demonstrates how llms.txt can go beyond simple description to actively guide AI behavior when discussing your product.

Common Mistakes to Avoid

After reviewing hundreds of llms.txt implementations, these are the most frequent errors:

• Vague entity definitions: "We help businesses grow" tells AI nothing. Be specific about your industry, product type, and target audience.
• Too many links: Including every page on your site defeats the purpose. Curate. AI systems perform better with 15 to 30 focused links than 200 uncategorized ones.
• Marketing copy instead of facts: AI systems need factual statements, not sales pitches. "The world's best platform" is noise. "An open-source database platform used by 500,000 developers" is signal.
• Missing descriptions: Links without descriptions force AI systems to visit each page to understand it. Add the colon and a brief explanation after every link.
• No Optional section: Failing to differentiate between core and supplementary content means AI systems treat everything with equal weight.

Next Steps

If you do not have an llms.txt file yet, use our free generator to build one in minutes. If you already have one, review it against the structure outlined above and check whether your entity definition is specific enough for AI systems to accurately represent your brand.

For more on the technical format details, see our llms.txt format specification guide. To understand how llms.txt fits into a broader AI optimization strategy, read about schema markup for AI SEO and entity authority for AI visibility.

Ready to get started? Build your llms.txt now and give AI systems the context they need to recommend your brand accurately.