Is llms.txt the same as robots.txt or sitemap.xml?

No. robots.txt tells crawlers what they can and cannot access. sitemap.xml is an XML list of every URL on your site for traditional search engines. llms.txt is a curated, human-readable Markdown summary of your most important content for AI systems. The three files complement each other and serve different purposes. robots.txt controls access. sitemap.xml is exhaustive. llms.txt is selective.

Do I actually need llms.txt?

Adoption of llms.txt is growing but not yet universal. Anthropic, Stripe, Mintlify, and Vercel all publish llms.txt files. Major AI engines do not yet treat it as a required ranking signal. However, publishing one costs nothing, signals deliberate AI-friendliness, and gives crawlers a fast map of your best content. The marginal upside is positive and the cost is zero.

Where exactly do I put the file?

At the root of your domain. The file must be accessible at https://yoursite.com/llms.txt — not inside any subfolder. For WordPress, upload via FTP or use a static-files plugin. For Webflow, Framer, and Wix, use the static files or 404 redirect feature. For Next.js or other frameworks, place it in the public/ or static/ directory.

What is the difference between llms.txt and llms-full.txt?

llms.txt is the curated short version with up to 50-60 of your most important URLs organized by section. llms-full.txt is the verbose companion that lists every URL on your site (similar to a sitemap but in Markdown). For larger sites, publishing both is recommended. AI engines prefer the curated llms.txt and fall back to llms-full.txt for deeper indexing.

How is llms.txt structured?

Every llms.txt starts with an H1 (# Project Name) followed optionally by a blockquote summary (> One-line description). The body consists of H2 sections (## Section Name) containing list items in the format `- [Title](URL): Description`. A common section is `## Optional` for less-critical links that AI engines should treat as lower priority.

How often should I update my llms.txt?

Re-generate every 1-3 months or whenever you publish a significant new content section, change your site structure, or launch new products. AI crawlers re-fetch the file periodically — typically every 1-7 days for major sites. There is no penalty for updating frequently.

Can I block AI engines from reading my llms.txt?

Yes, the same way you would block any file. Add a Disallow rule for /llms.txt in your robots.txt for specific user agents. However, the entire purpose of llms.txt is to help AI engines understand your site. Blocking it defeats the purpose. If you want to control AI access, use robots.txt at the user-agent level instead.

Will llms.txt help me rank in ChatGPT or Perplexity?

Indirectly, yes. AI engines that read llms.txt will discover and index your most important content faster. This means more of your pages get considered for citation. llms.txt by itself is not a ranking signal in the traditional SEO sense — content quality, schema markup, and E-E-A-T still matter most. But it removes a discoverability barrier and makes the rest of your GEO work pay off faster.

GEO Guide

llms.txt: the complete guide

Q: How do I know if my llms.txt is valid?

Use the GrandRanker llms.txt Validator. It runs 12 spec rule checks including: H1 presence and content, summary blockquote, H2 sections, link format, absolute URLs, file encoding, line length, and more. The validator outputs a 0-100 score with specific fix recommendations.

11 min read·Updated May 9, 2026

llms.txt is the emerging open standard for helping AI engines understand your website. This guide covers the official spec, where to host the file, how to write each section, and the two free tools that automate the entire workflow.

Quick answer

llms.txt is a Markdown file at the root of your domain (yoursite.com/llms.txt) that lists your most important URLs in a curated, AI-readable format. It was proposed by Jeremy Howard in late 2024 as an LLM-friendly companion to robots.txt and sitemap.xml. Anthropic, Stripe, Mintlify, and Vercel publish one. You should too — it takes 30 seconds with the right generator.

What is llms.txt?

llms.txt is a single Markdown file served at the root of your domain (yoursite.com/llms.txt) that gives AI systems a curated map of your most important content. Unlike robots.txt (which controls access) or sitemap.xml (which lists every URL), llms.txt is selective and human-readable.

The standard was proposed by Jeremy Howard, co-founder of Answer.AI and fast.ai, in llmstxt.org in September 2024. The motivation: large language models like ChatGPT, Claude, and Perplexity have limited context windows. They cannot crawl an entire 10,000-page website to answer a single user question. They need a fast, curated entry point.

Think of llms.txt as your website's elevator pitch for AI engines. Where robots.txt says "yes" or "no", llms.txt says "here is what matters most".

The spec

The official llms.txt structure

Every spec-compliant llms.txt has the same five-part structure. Get any of these wrong and AI engines may parse your file incorrectly or skip it entirely.

H1 with project name

A single # heading at the top of the file. This is the canonical name of your project, brand, or site.

# GrandRanker

Optional summary blockquote

A `>` blockquote immediately after the H1. One sentence describing what your site does.

> AI-powered SEO platform that automates the full content lifecycle.

Optional description paragraphs

Plain prose between the blockquote and the first H2. Use this for context, philosophy, or technical details.

GrandRanker handles the full content lifecycle from a single keyword.

H2 sections with link items

Each section uses `##` and contains link items in the format `- [Title](URL): Description`. URLs must be absolute.

## Documentation

- [Getting Started](https://grandranker.com/start): Quick setup

Optional section

A special H2 named `## Optional`. AI engines treat these as lower priority than other sections.

## Optional

- [Affiliate Program](https://grandranker.com/affiliate-program): 30% recurring

Complete example

# GrandRanker

> AI-powered SEO platform that automates keyword research, content creation, optimization, and publishing.

GrandRanker handles the full content lifecycle from a single seed keyword: research, drafting, on-page optimization, internal linking, and one-click publishing to WordPress, Shopify, Webflow, Wix, and Framer.

## Documentation

- [Getting Started](https://grandranker.com/start): Set up your first article
- [Pricing](https://grandranker.com/pricing): Plans starting from $49/month
- [Platform overview](https://grandranker.com/platform): CMS integrations

## Free Tools

- [GEO Score Checker](https://grandranker.com/free-tools/geo-score-checker): Score any URL 0-100
- [llms.txt Validator](https://grandranker.com/free-tools/llms-txt-validator): Validate against the spec
- [llms.txt Generator](https://grandranker.com/free-tools/llms-txt-generator): Auto-generate a file

## Optional

- [Affiliate Program](https://grandranker.com/affiliate-program): 30% recurring commission

Why llms.txt matters in 2026

AI search adoption is no longer theoretical. ChatGPT processes over 800 million weekly users. Google AI Overviews appear on roughly 30% of US search queries. Perplexity, Gemini, and Claude all run their own crawlers and synthesize answers from web content.

These systems do not return ten blue links. They synthesize a single answer and cite three to five sources. Being one of those cited sources is the new "ranking on page one". And before you can be cited, you have to be efficiently discovered.

llms.txt solves the discovery problem at the structural level. Instead of asking AI engines to crawl your entire site to figure out what matters, you tell them directly. The file is small enough to fit inside any LLM context window and clear enough that downstream processing is reliable.

Faster AI discovery

AI crawlers fetch llms.txt first and use it to prioritize which pages to fetch next. Skip the wait for full-site crawls.

Deliberate AI-friendliness

Publishing the file signals to AI systems and the broader ecosystem that you actively support AI search.

Curation, not exhaustion

Sitemap.xml lists everything. llms.txt lists what matters. Quality over quantity is what AI engines reward.

Zero downside

Costs nothing to publish. No SEO penalty. No conflict with robots.txt or sitemap.xml. Pure upside.

Who is already publishing llms.txt

Adoption is accelerating among technically sophisticated companies. As of mid-2026, the following companies publish a public llms.txt:

Anthropic

Documentation index for Claude API

docs.anthropic.com/llms.txt

Stripe

Comprehensive developer docs map

docs.stripe.com/llms.txt

Mintlify

Documentation platform reference

mintlify.com/docs/llms.txt

Vercel

Deployment platform docs

vercel.com/docs/llms.txt

Cursor

AI code editor documentation

docs.cursor.com/llms.txt

GrandRanker

AI SEO platform

grandranker.com/llms.txt

The pattern is clear: technical platforms publish first because their users are most likely to be querying AI engines for documentation.

Free tools

Generate and validate yours in 30 seconds

Two free tools that automate the whole workflow. Use the generator first to build the file, then the validator to confirm it meets the spec.

Start here

llms.txt Generator

Crawls your homepage and sitemap, deduplicates and categorizes internal links by URL pattern (Documentation, Product, Blog, etc.), and outputs a spec-compliant Markdown file you can edit and download. From URL to ready-to-deploy llms.txt in 30 seconds.

Generate llms.txt

llms.txt Validator

Validates your llms.txt against the official spec. Runs 12 rule checks: H1 structure, summary blockquote, H2 sections, link format, absolute URLs, file encoding, line length, and more. Outputs a 0-100 score with line-numbered fixes and an embeddable verification badge.

Validate llms.txt

Writing your own llms.txt manually

If you prefer to write the file by hand instead of using the generator, follow these six steps. The whole process takes about 15 minutes for a typical 50-100 page site.

Create the file

Open your text editor. Create a new file named exactly llms.txt (lowercase, no extension changes). Save it with UTF-8 encoding.

Add the H1 with your project name

First line: a single # followed by your brand or project name. Avoid emojis or special characters that might confuse Markdown parsers.

# Your Site Name

Add the summary blockquote

Leave a blank line, then add a > blockquote with a one-sentence summary. Aim for 80-160 characters that explain what your site does.

> A clear, concise description of your site or product.

Group your URLs into 3-6 H2 sections

Common sections: Documentation, Product, Free Tools, Pricing, Blog, Help & Support. Each H2 is "## Section Name" with link items underneath.

## Documentation

- [Getting Started](https://yoursite.com/start): One-line description
- [API Reference](https://yoursite.com/api): API documentation

Add an Optional section for de-prioritized links

Use the special "## Optional" section name for less-critical pages: legal pages, archive content, secondary tools. AI engines treat these with lower priority.

## Optional

- [Privacy Policy](https://yoursite.com/privacy): Privacy statement
- [Terms of Service](https://yoursite.com/terms): Legal terms

Upload to your domain root

The file must be reachable at https://yoursite.com/llms.txt. WordPress: FTP into your site root. Webflow/Framer/Wix: use the static files or 404 redirect feature. Static hosts (Vercel, Netlify, Cloudflare Pages): place in public/ or static/ directory.

Common pitfalls

5 most common llms.txt mistakes

Most invalid llms.txt files share the same handful of errors. Avoid these.

Wrong file location

Wrong: yoursite.com/docs/llms.txt or yoursite.com/static/llms.txt

Right: yoursite.com/llms.txt — must be at the domain root

Relative URLs in link items

Wrong: - [Pricing](/pricing): Plans and pricing

Right: - [Pricing](https://yoursite.com/pricing): Plans and pricing

Multiple H1 headers

Wrong: Two or more # lines at the top of the file

Right: Exactly one # for the project name. Use ## for sections.

Missing summary or H2 sections

Wrong: Just a # title with a list of links and nothing else

Right: H1 + > blockquote summary + 2-5 ## sections with link items

Listing every URL on the site

Wrong: Pasting every page from your sitemap (200+ links)

Right: Curate to 30-60 most important URLs. Use llms-full.txt for the exhaustive version.

Continue learning

Other essential GEO guides.

GEO Pillar Guide

The complete 2026 guide to Generative Engine Optimization.

Read guide

AI Citability

How to write paragraphs that AI engines actually quote.

Read guide

Schema for AI

JSON-LD bundles optimized for AI extraction.

Read guide

Frequently asked

Everything to know about llms.txt.

llms.txt is a Markdown file at the root of your domain (yoursite.com/llms.txt) that lists your most important URLs in a curated, AI-friendly format. The standard was proposed by Jeremy Howard (founder of Answer.AI and fast.ai) in late 2024. It is designed to help large language models like ChatGPT, Claude, and Perplexity understand your website structure without crawling every page.

Ready to ship a complete GEO strategy?

GrandRanker writes AI-citation-ready content with proper structure, schema, and citations. From keyword to ranked article in 10 minutes.

Generate llms.txt now