Complete Guide Updated 2025 · 15 min read

What is the llms.txt file, and how do you create one?

The definitive guide to making your website visible to AI. With real examples from Anthropic, Cursor, and more—plus a free generator that creates your file in seconds.

4,000+ websites generated
36M+ AI searches by 2028
Share:

TL;DR

The llms.txt file is a Markdown document that helps AI systems (ChatGPT, Claude, Perplexity) understand your website. Place it at yoursite.com/llms.txt and AI can instantly know what your business does, what pages matter, and how to accurately describe you.

# What is llms.txt?

The llms.txt file is a standardized plain text document that helps artificial intelligence systems understand your website's content, structure, and purpose. Think of it as a "cheat sheet" for AI—a concise summary that tells ChatGPT, Claude, and other large language models exactly what your business does and where to find your most important content.

The standard was proposed by Jeremy Howard, founder of fast.ai and Answer.AI, as a solution to a growing problem: AI systems struggle to parse complex websites. They get lost in navigation menus, miss important pages, and often misrepresent businesses in their responses.

The file in a nutshell

  • • Lives at: yourdomain.com/llms.txt
  • • Format: Plain text with Markdown syntax
  • • Contains: Business description + page URLs + descriptions
  • • Purpose: Help AI understand and accurately represent your site

Without an llms.txt file, AI systems must crawl your entire website, parse complex HTML, and guess what matters. This often leads to AI hallucinations—where AI invents information about your business that isn't true—or simply ignores your site entirely in favor of competitors who have made their content more accessible. You can create llms.txt in seconds to prevent this.

# Live examples from real websites

Want to see what llms.txt files look like in practice? Here are real examples from companies that have already implemented the standard:

Pro tip: Check any website's llms.txt by adding /llms.txt to the domain. Try it with your favorite sites!

# File structure & syntax

An llms.txt file uses Markdown syntax—a simple formatting language that's readable by both humans and machines. Here's the complete structure:

llms.txt template
# Your Website Name

> A comprehensive description of your business, products, 
> or services. This should be 2-4 sentences that capture 
> your core value proposition and target audience.

## Main Pages

- [Homepage](https://yoursite.com/): Brief description of 
  what visitors find on your main page.
- [Products](https://yoursite.com/products): Overview of 
  your product offerings with key features.
- [Pricing](https://yoursite.com/pricing): Pricing tiers 
  and what's included in each plan.
- [Documentation](https://yoursite.com/docs): Technical 
  guides and API reference.

## Resources

- [Blog](https://yoursite.com/blog): Industry insights, 
  tutorials, and company updates.
- [Case Studies](https://yoursite.com/cases): Success 
  stories from customers.

## Optional

- [Additional links that are less critical]

Key elements explained

#

Title (H1)

Your website or company name. Only one per file.

>

Blockquote (Description)

Your business description. This is critical—it's the context AI uses to understand everything else.

##

Section Headers (H2)

Organize your pages into logical groups like "Products", "Resources", "Support".

- []()

Page Links

Markdown links with descriptions: - [Title](URL): Description

Real-world example

Example: SaaS analytics company
# DataPulse Analytics

> DataPulse provides real-time business intelligence dashboards 
> for e-commerce companies. We help online retailers track sales, 
> inventory, and customer behavior with AI-powered insights. 
> Founded in 2022, we serve 500+ businesses from startups to 
> enterprise retailers.

## Product

- [Features](https://datapulse.io/features): Real-time dashboards, 
  AI anomaly detection, custom reports, and 50+ integrations 
  including Shopify, WooCommerce, and BigCommerce.
- [Pricing](https://datapulse.io/pricing): Three tiers - Starter 
  ($29/mo for up to 10k orders), Growth ($99/mo for 100k orders), 
  and Enterprise (custom pricing with dedicated support).
- [Integrations](https://datapulse.io/integrations): Native 
  connections to major e-commerce platforms, payment processors, 
  and marketing tools.

## Resources

- [Documentation](https://docs.datapulse.io/): API reference, 
  setup guides, and webhook documentation for developers.
- [Blog](https://datapulse.io/blog): E-commerce analytics tips, 
  industry benchmarks, and product updates.
- [Case Studies](https://datapulse.io/customers): How companies 
  like Acme Store increased revenue 34% using our insights.

## Company

- [About](https://datapulse.io/about): Our story, team, and 
  mission to democratize e-commerce analytics.
- [Careers](https://datapulse.io/careers): Open positions in 
  engineering, sales, and customer success.

# Why llms.txt matters for AI visibility

The way people find information is undergoing its biggest shift since Google. Instead of searching, clicking links, and reading—users now ask AI directly and expect complete answers without leaving the chat.

36M+

US adults will use AI for search by 2028

2x

Growth in AI search usage since 2024

65%

Of Gen Z prefers AI over traditional search

This shift has created a new field: Generative Engine Optimization (GEO)—optimizing your content so AI systems can accurately understand and cite your website. The llms.txt file is the foundation of AI SEO.

The reality: When someone asks ChatGPT about a topic your business covers, you want to be in that answer. Without proper AI visibility optimization, competitors with llms.txt files get cited instead—regardless of how good your actual content is. Try the free llmstxt generator tool to fix this.

What happens without llms.txt

AI hallucinations: AI invents features you don't have or prices that are wrong

Missed opportunities: AI recommends competitors because it can't parse your site

Outdated information: AI cites old content because it can't find what's current

What happens with llms.txt

Accurate representation: AI describes your business exactly as you intend

Higher visibility: AI can confidently recommend you for relevant queries

Competitive edge: You're ahead of the 99% of sites without llms.txt

# How AI systems use llms.txt

Large language models process text to understand meaning and context. When AI encounters your website, it needs to quickly determine what you do and which pages matter most. Here's the process:

1

AI requests your llms.txt

When processing queries about your industry, AI checks if yourdomain.com/llms.txt exists

2

Parses your business context

The description block tells AI what you do, who you serve, and your key differentiators

3

Indexes page content

Each URL with description gets added to AI's understanding of your site

4

Cites you accurately

When users ask relevant questions, AI can confidently recommend and describe your offerings

🎯 Key insight

The llms.txt file doesn't guarantee AI will cite you—it ensures that when AI does talk about you, the information is accurate. It's the difference between AI saying "I think they might offer X" versus "They offer X, Y, and Z with pricing starting at $29/month." Use our llms.txt generator to create your file in seconds.

# How to create an llms.txt file

You have three options for creating your llms.txt file. The fastest is to generate your full llms.txt file online here. Choose based on your technical comfort level and how much customization you want:

RECOMMENDED

Method 1: Use our free LLM txt generator

The fastest way. Our free llms.txt generator online automatically scans your sitemap, analyzes each page, and this LLM text generator creates llms full documentation with AI-optimized descriptions. Takes about 30 seconds.

  1. 1 Go to llmstxtgenerator.org
  2. 2 Enter your website URL
  3. 3 Wait ~30 seconds for AI analysis
  4. 4 Download your llms.txt file
  5. 5 Upload to your website root directory
Generate Your llms.txt Free →

First 20 URLs free · No signup required

Method 2: Create manually

If you prefer full control, create the file yourself using any text editor:

  1. 1 Create a new file called llms.txt
  2. 2 Add your site name as a heading: # Your Site Name
  3. 3 Write a 2-4 sentence description using blockquote syntax: > Your description
  4. 4 List each important page with URL and description
  5. 5 Upload to your website's root directory

Method 3: AI-assisted writing

Use ChatGPT or Claude to help write your descriptions. Here's a prompt you can use:

I need to create an llms.txt file for my website [YOUR SITE].

Here are my main pages:
- Homepage: [URL]
- Product/Service page: [URL]
- Pricing: [URL]
- Blog: [URL]

Please generate an llms.txt file with:
1. A compelling 3-4 sentence business description
2. Detailed descriptions for each page (2-3 sentences each)
3. Proper Markdown formatting

My business does: [BRIEF DESCRIPTION]
My target audience is: [TARGET AUDIENCE]

# Platform-specific guides

Here's how to upload your llms.txt file on popular website platforms:

📝

WordPress

  1. 1. Access via FTP or File Manager
  2. 2. Navigate to /public_html/
  3. 3. Upload llms.txt file
  4. 4. Verify at yoursite.com/llms.txt
🛒

Shopify

  1. 1. Go to Online Store → Themes
  2. 2. Edit code → Assets folder
  3. 3. Add new file: llms.txt
  4. 4. Create redirect to /llms.txt

Wix

  1. 1. Dashboard → Settings → SEO
  2. 2. Add file to site files
  3. 3. Or use a static page
  4. 4. Set URL as /llms.txt
◼️

Squarespace

  1. 1. Create a new page
  2. 2. Add Code Block with content
  3. 3. Set URL slug to /llms.txt
  4. 4. Hide from navigation
🌊

Webflow

  1. 1. Project Settings → Hosting
  2. 2. Add custom file to assets
  3. 3. Or create static page
  4. 4. Publish and verify
⚙️

Custom / Self-hosted

  1. 1. FTP to your server
  2. 2. Upload to document root
  3. 3. Usually /var/www/html/
  4. 4. Set correct permissions

# llms.txt vs robots.txt

Both files help machines understand your website, but they serve completely different purposes. Here's the breakdown:

llms.txt robots.txt
Purpose Describe content for AI understanding Control search engine crawling access
Target audience AI systems (ChatGPT, Claude, etc.) Search engine bots (Google, Bing)
Format Markdown syntax Plain text directives
Content URLs + descriptions + business context Allow/disallow rules for crawlers
Goal Get featured accurately in AI responses Control which pages appear in search

You need both files. robots.txt tells search engines what to index, while llms.txt tells AI how to understand what's indexed. They work together, not as alternatives. You can create your llms.txt file online while keeping your existing robots.txt. For a deeper comparison, see our guide: llms.txt vs robots.txt vs sitemap.xml.

# Best practices

✓ Write detailed, specific descriptions

Don't just list page titles. Include 2-3 sentences per page explaining the content, audience, and value. More context = better AI understanding.

✓ Include concrete details

Mention specific pricing, features, integrations, and numbers. "Starting at $29/month" is better than "affordable pricing."

✓ Prioritize important pages first

List your most valuable content at the top. AI systems often weight earlier content as more important.

✓ Keep it current

Update your llms.txt when you add pages, change pricing, or update products. Outdated info leads to AI mistakes. Our free llms.txt generator makes re-generating quick and easy.

✓ Use natural language

Write descriptions as if explaining to a smart colleague. Avoid jargon or marketing fluff that could confuse AI.

✗ Don't include sensitive pages

Skip admin areas, internal tools, or confidential content. Only include publicly accessible pages you want AI to know about.

# Troubleshooting

My llms.txt returns a 404 error

Check these common causes:

  • • File isn't in the root directory (should be alongside index.html)
  • • Filename has wrong extension (must be .txt, not .txt.txt)
  • • Server doesn't serve .txt files (check server config)
  • • CDN is caching old version (clear cache)
The file shows as download instead of displaying

Your server may not have the correct MIME type for .txt files. Add this to your server config:

Apache (.htaccess): AddType text/plain .txt

Nginx: types { text/plain txt; }

AI still gets my business wrong after adding llms.txt
  • • AI models are trained periodically—it may take time for updates to reflect
  • • Make your description more explicit and detailed
  • • Include specific facts, not vague claims
  • • Ensure your sitemap.xml is also up to date
  • • Try regenerating your llms.txt with more detailed content
How do I know if AI is reading my llms.txt?

Currently there's no way to directly track llms.txt requests since AI crawlers don't always identify themselves. However:

  • • Check your server logs for requests to /llms.txt
  • • Test by asking AI about your business and see if answers improve
  • • Major AI providers are adopting the standard—presence helps even without tracking

# Frequently asked questions

Is llms.txt an official standard?

It's a proposed standard by Jeremy Howard of Answer.AI. While not officially endorsed by all AI providers, it's been adopted by major companies including Anthropic (Claude), Stripe, and Cursor. It's becoming the de facto standard for AI optimization.

Do ChatGPT and Claude automatically read llms.txt?

AI systems are increasingly supporting the standard. ChatGPT's browsing feature and Claude's web access can read llms.txt files. As adoption grows, support will become more universal. Having the file ready positions you for when it's fully integrated.

How often should I update my llms.txt?

Update it whenever you: add new pages, change pricing, update features, rebrand, or remove old content. At minimum, review quarterly to ensure accuracy. You can quickly regenerate it using our online tool anytime.

Can llms.txt hurt my traditional SEO?

No. llms.txt is a separate file that doesn't affect how search engines crawl or rank your site. It only helps AI systems understand your content better. Traditional SEO and GEO work alongside each other.

How many pages should I include?

Include all publicly accessible pages that are important for understanding your business. For small sites, that might be 10-20 pages. For large sites with thousands of pages, focus on the most important categories and key pages within each.

Is there a maximum file size?

There's no official limit, but keep it reasonable. AI systems have context windows—a file with thousands of pages may not be fully processed. Focus on quality descriptions for your most important content rather than listing everything.

Related Articles

Try Our Free LLMs.txt Generator Online

Use our LLM txt generator to create llms full documentation in seconds. This free llms.txt generator scans your sitemap and our LLM text generator creates AI-optimized descriptions for every page.

Free LLMs.txt Generator

No signup required · First 20 URLs free · Used by 4,000+ websites