How do I get found in AI like I used to get found in SEO? That or something similar is a question a lot of people are asking these days. Working to figure out how to get found in AI, what some are calling GEO, is not really as clear as SEO.

As in, SEO primarily focused on organic Google results, and practices were refined over a decade like link building, citations, keyword-rich content, quality metrics, speed scores, and so on. Not so much with AI.

Thankfully we do have some gold treasure to pursue deep in the mines of the AI discovery caverns, to at least get a little traction with the ole AI bots. And today we’ll discuss one of those little nuggets, something called the LLMS.txt file.

Remember when having a sitemap.xml file was all you needed to get Google to crawl and rank your website? Those days shaped how we thought about organic visibility for nearly two decades. But the landscape has fundamentally changed.

We’re no longer optimizing for just one search engine. Today, your potential customers are asking ChatGPT for recommendations, querying Claude for technical documentation, consulting Gemini for research, and using Perplexity to discover new products. The fragmentation is real, and it’s only accelerating.

Welcome to the era of Generative Engine Optimization (GEO), where visibility isn’t about ranking on page one—it’s about being cited in the answer itself. And just like sitemap.xml became foundational for SEO, llms.txt is becoming foundational for GEO.

The Shift from SEO to GEO: Why Everything Changed

Traditional search engine optimization focused on one primary goal: getting your link to appear in the top 10 results on Google’s search engine results page (SERP). You optimized title tags, built backlinks, improved page speed, and hoped to climb the rankings.

But generative AI has fundamentally altered how people find information.

According to recent data, AI-referred sessions jumped 527% between January and May 2025. People aren’t just clicking through ten blue links anymore—they’re receiving synthesized answers from AI platforms that cite only 2-7 sources on average, compared to Google’s traditional 10 results.

This creates an entirely new challenge. In the old world, if you weren’t in the top 10, you could still get traffic by ranking 11-20. In the AI world, if you’re not one of the 2-7 cited sources, you effectively don’t exist.

Understanding AI Fragmentation: You Can’t Just “Show Up in Google” Anymore

Here’s the uncomfortable truth that many marketers are still grappling with: Google is no longer the only game in town.

Users are now distributed across multiple AI platforms, each with its own way of sourcing and synthesizing information:

ChatGPT has become the go-to for conversational queries and problem-solving. Research shows ChatGPT cites Wikipedia 47.9% of the time when answering factual questions, followed by news sites and educational resources.

Perplexity AI emphasizes real-time information and processes over 500 million queries monthly. Perplexity AI recorded 153 million website visits in May 2025, up from 52.4 million visits in March 2024, representing 191.9% growth in a year.

Claude is increasingly used by developers and technical teams for documentation research and code assistance.

Gemini (Google’s AI) is integrated directly into Google Search and workspace tools, creating another visibility layer you need to consider.

Grok serves X (Twitter) users with real-time information and trending topics.

Each of these platforms has different crawling behaviors, different citation preferences, and different ways of determining which sources to trust. You can’t optimize for just one and hope the rest follow. You need a strategy that works across all of them.

This is where llms.txt comes in.

What Exactly Is LLMS.TXT?

Think of llms.txt as the modern equivalent of sitemap.xml, but specifically designed for AI consumption instead of traditional search crawlers.

The llms.txt proposal was created to address a critical challenge: context windows in large language models are too small to handle most websites in their entirety, and converting complex HTML pages with navigation, ads, and JavaScript into LLM-friendly plain text is both difficult and imprecise.

The solution? A simple markdown file placed at your website’s root (yoursite.com/llms.txt) that provides AI systems with:

• A clear overview of your site or project
• Structured links to your most important content
• Brief descriptions of what each resource contains
• Optional sections for supplementary materials

Unlike robots.txt, which tells crawlers what NOT to access, llms.txt tells AI systems what SHOULD be accessed and why it matters.

The Anatomy of a Perfect LLMS.TXT File

A properly formatted llms.txt file contains specific sections in a particular order: an H1 heading with the project or site name (the only required section), a blockquote with a short summary containing key information, zero or more markdown sections with detailed information, and organized documentation links.

Here’s what a well-structured llms.txt looks like in practice:

# YourCompany Product Documentation

> YourCompany provides cloud-based analytics solutions for enterprise teams, helping organizations make data-driven decisions with real-time insights and AI-powered recommendations.

Important notes:
- Our API uses OAuth 2.0 authentication
- All endpoints return JSON responses
- Rate limits apply to free tier accounts

## Core Documentation
- [Getting Started](https://docs.yourcompany.com/start): Quick setup guide for new users
- [API Reference](https://docs.yourcompany.com/api): Complete API endpoint documentation
- [Authentication Guide](https://docs.yourcompany.com/auth): Security and authentication methods

## Integration Guides
- [Python SDK](https://docs.yourcompany.com/python): Python integration examples
- [JavaScript SDK](https://docs.yourcompany.com/js): JavaScript and Node.js integration
- [REST API](https://docs.yourcompany.com/rest): Direct REST API usage

## Optional Resources
- [Changelog](https://docs.yourcompany.com/changelog): Recent updates and releases
- [Community Forum](https://community.yourcompany.com): User discussions and support

Notice how each link includes a brief, actionable description. This isn’t just helpful for humans—it’s critical for AI systems trying to determine which resource answers a specific query.

Real-World Examples: Who’s Already Winning with LLMS.TXT

Several leading companies have already implemented llms.txt to great effect. Let’s examine what we can learn from them:

Anthropic (the company behind Claude) has implemented a comprehensive llms.txt file across their documentation. Their implementation demonstrates best practices for technical documentation sites, with clear categorization by topic and detailed descriptions for each resource.

Zapier takes an API-focused approach. Zapier’s llms.txt centers around documentation for their AI Actions API, providing a well-organized list of endpoints. For an API-first company, this makes perfect sense—they’re optimizing for the exact queries developers will ask AI assistants.

Stripe organizes their llms.txt by product categories, acknowledging that they offer multiple distinct products. This hierarchical approach helps AI systems understand the relationships between different pieces of content.

The best implementations create a logical hierarchy that helps AI systems understand the relationships between different pieces of content, with API-focused companies organizing by endpoints, product-diverse companies organizing by product categories, and content-focused companies organizing by content type.

The Technical Implementation: Two Files You Need to Know

The llms.txt specification actually includes two distinct files, each serving a different purpose:

/llms.txt – Your Navigation Guide

The /llms.txt file is a streamlined view of your documentation navigation, containing the name of the project, a brief summary, and organized links to detailed markdown files.

This file acts as an index or table of contents. It’s lightweight and designed to be quickly parsed by AI systems to understand your site’s structure.

/llms-full.txt – Your Complete Context File

The llms-full.txt file combines your entire documentation site into a single file as context for AI tools.

This file contains the full text of your documentation, allowing users to paste a single URL into an AI chat interface and give it complete context about your product or service.

A key consideration when using llms-full.txt is its size—for extensive documentation, this file may become too large to fit into an LLM’s context window. For large documentation sites, the llms.txt approach with individual markdown files may be more practical.

Beyond LLMS.TXT: The .md Extension Strategy

The llms.txt proposal includes an additional recommendation that amplifies its effectiveness: providing a clean markdown version of pages at the same URL as the original page, but with .md appended.

For example, if you have a page at:

https://docs.yoursite.com/getting-started

You should also provide:

https://docs.yoursite.com/getting-started.md

This markdown version should contain the same information as your HTML page, but stripped of navigation, ads, JavaScript, and other elements that make it harder for AI systems to extract the core content.

This simple addition makes it dramatically easier for AI systems to consume your content. Instead of parsing complex HTML with nested divs and dynamic elements, they get clean, structured markdown that’s optimized for their processing pipelines.

How AI Systems Actually Use LLMS.TXT

Understanding how AI platforms leverage llms.txt helps you optimize more effectively.

When an AI system encounters a query it needs to answer, it goes through a process called Retrieval-Augmented Generation (RAG). This involves:

1. Understanding the query and determining what information is needed
2. Retrieving relevant documents from the web or its training data
3. Extracting key information from those documents
4. Synthesizing an answer that combines information from multiple sources
5. Citing sources that contributed to the answer

The llms.txt file makes steps 2 and 3 dramatically more efficient. Instead of crawling your entire site and trying to determine which pages are most relevant, AI systems can:

• Immediately understand your site’s structure from llms.txt
• Identify which specific resources are most likely to answer the query
• Access clean markdown versions of those resources
• Extract information with higher accuracy and less computational overhead

This means an AI assistant can immediately understand your documentation structure without crawling multiple pages or parsing complex navigation elements, while also filtering out ads, HTML markup, or JavaScript-rendered elements to optimize token usage.

WordPress Users: Automated LLMS.TXT Implementation

If you’re running WordPress, you’re in luck. Several plugins now automate llms.txt creation and management.

Yoast SEO has built-in llms.txt support. Users can enable llms.txt directly from plugin settings with automated file creation. The plugin intelligently determines which pages to include based on your existing SEO settings.

Yoast SEO includes the 5 categories or tags with the most content pieces attached to them, and custom post types are included if the “Show tags in search results” box is ticked in the Search Appearance settings.

Hostinger Plugin offers similar functionality for Hostinger WordPress hosting customers, with a simple toggle to turn on llms.txt generation.

The key advantage of using these plugins is that they keep your llms.txt file automatically updated as you publish new content, ensuring AI systems always have access to your latest pages.

Creating Your LLMS.TXT File: A Step-by-Step Implementation Guide

Ready to implement llms.txt on your own site? Here’s exactly how to do it:

Step 1: Audit Your Most Important Content

Start by identifying which pages on your site provide the most value to visitors. These typically include:

• Product or service overview pages
• Getting started guides
• API documentation
• Frequently asked questions
• Case studies or success stories
• Pricing information
• Key blog posts or resources

Don’t try to include everything. The llms.txt file should provide concise, expert-level information gathered in a single, accessible location. Quality over quantity is crucial here.

Step 2: Organize Content by Category

Group your important pages into logical categories. Consider how someone unfamiliar with your site would want information organized:

• Documentation – Technical guides and references
• Tutorials – Step-by-step instructions
• Product Information – Features, pricing, comparisons
• Resources – Additional learning materials
• Support – FAQs, troubleshooting, contact info

Step 3: Write Descriptive Link Titles

For each link in your llms.txt, write a brief (5-15 word) description that clearly explains what the page contains and why it’s valuable. These descriptions help AI systems determine relevance to specific queries.

Weak: [API Docs](https://site.com/api) Better: [API Reference](https://site.com/api): Complete REST API documentation with authentication and rate limits

Step 4: Create the File

Open a text editor and create your llms.txt file following the markdown structure outlined earlier. Remember:

• Start with an H1 heading (your site/product name)
• Add a blockquote with a concise summary
• Include any important context or notes
• Organize links under H2 section headings
• Use an “Optional” section for supplementary resources

Step 5: Upload to Your Site Root

Upload the completed llms.txt file to your website’s root directory, making it accessible at yoursite.com/llms.txt.

For most websites, this means placing it in the same directory as your index.html or homepage file.

Step 6: Validate and Test

Visit yoursite.com/llms.txt in your browser to confirm it’s publicly accessible. You should see the raw markdown content.

Test it by copying your llms.txt URL into ChatGPT or Claude and asking questions about your site. The AI should be able to navigate your content more effectively.

Step 7: Create Markdown Versions of Key Pages

For your most important pages, create .md versions as described earlier. These should contain the same information as your HTML pages but in clean markdown format.

If you use a static site generator or documentation platform, many now support this automatically.

Tools and Generators to Speed Up Implementation

Several tools can help you create llms.txt files faster:

Mintlify automatically generates llms.txt files for documentation sites and keeps them synchronized with your content. Mintlify automatically hosts an llms.txt file at the root of your project that lists all available pages in your documentation, and this file is always up to date and requires zero maintenance.

WordLift’s LLMs.txt Generator allows you to enter your website address, choose pages to include, and generates properly formatted markdown that you can download and upload.

Firecrawl offers a tool that scrapes your website and automatically creates the llms.txt file based on your content structure.

GitBook provides built-in llms.txt generation for documentation sites, keeping files synchronized with your content updates.

These tools are particularly valuable for larger sites where manually creating and maintaining llms.txt would be time-consuming.

Measuring Success: What to Expect After Implementation

Once you’ve implemented llms.txt, what results should you expect?

Immediate Effects (1-4 Weeks)

You won’t see instant results. AI systems need time to recrawl your site and update their knowledge. However, within a few weeks you should notice:

• More accurate AI-generated descriptions of your product or service
• Better citation rates when people ask AI about topics you cover
• Improved relevance in AI-generated recommendations

Medium-Term Results (1-3 Months)

Research shows that businesses implementing GEO strategies see citation increases of up to 40% based on studies from Princeton University and Georgia Tech.

You may observe:

• Increased traffic from AI-referred sessions
• Higher quality leads who are better informed about your offerings
• Reduced support inquiries as AI systems provide more accurate preliminary information

Long-Term Benefits (3+ Months)

The compounding nature of AI citations means early adopters gain significant advantages. As AI systems repeatedly cite authoritative sources, those sources become even more likely to be cited in the future.

Early adoption of GEO practices creates authority signals that compound over time, building stronger positions in AI recommendation systems.

Common Mistakes to Avoid

Through analyzing hundreds of llms.txt implementations, several common pitfalls have emerged:

Including Too Much Content – Don’t try to list every page on your site. Focus on your most valuable 20-50 pages. More isn’t better here.

Vague Descriptions – Generic descriptions like “Learn more about our product” don’t help AI systems determine relevance. Be specific about what each page contains.

Ignoring Updates – Your llms.txt file should evolve with your site. Set a quarterly reminder to review and update it as you publish new content.

Forgetting the Blockquote Summary – The summary at the top of your llms.txt is often the first thing AI systems read. Make it count by clearly explaining what your site/product does.

Not Creating .md Versions – The llms.txt file alone helps, but combining it with markdown versions of your pages maximizes effectiveness.

The Broader GEO Strategy: LLMS.TXT Is Just the Beginning

While llms.txt is foundational, it’s part of a larger Generative Engine Optimization strategy that includes:

Structured Data Markup – Schema.org markup helps AI systems understand your content’s context and meaning. Implement structured data for products, articles, FAQs, and other relevant content types.

Authority Building – AI systems exhibit a systematic bias towards earned media (third-party, authoritative sources) over brand-owned content. Building citations and mentions on authoritative third-party sites increases your chances of being cited by AI.

Content Optimization – Structure content with direct answers in the first 40-60 words, maintain fact density with statistics every 150-200 words, and cite authoritative sources throughout.

Regular Content Updates – AI systems favor fresh, current information. Regularly updating your content signals that it’s actively maintained and trustworthy.

Platform-Specific Tactics – ChatGPT favors encyclopedic content, Perplexity rewards recency and community examples, and Google AI Overviews prioritize existing top-ranking content.

Addressing the Skeptics: Is LLMS.TXT Worth It?

Some marketers remain skeptical about llms.txt. The most common objection is that no major AI company has officially endorsed the standard.

Here’s why that criticism misses the point:

The strongest use cases for llms.txt aren’t about broad visibility that traditional SEO targets—they’re about context awareness and usability. Companies implementing llms.txt see value not from discovery, but from improved accuracy when AI systems reference their content.

Developer-focused companies are already seeing value in using llms.txt to help users get the right documentation suggestions and API examples from AI coding tools while reducing token usage.

The question isn’t whether llms.txt will become a universal standard—it’s whether you want to make it easier for AI systems to accurately represent your content. Given the minimal implementation effort and the rapid growth of AI-referred traffic, the answer for most sites is yes.

Future-Proofing Your AI Visibility

The AI landscape will continue evolving rapidly. New models will emerge, existing platforms will change their algorithms, and user behavior will shift.

But the fundamental principle behind llms.txt will remain constant: AI systems perform better when given clean, structured, well-organized information.

By implementing llms.txt now, you’re not just optimizing for today’s AI platforms—you’re building a foundation that will serve you well regardless of which AI systems dominate in the future.

Your Action Plan: Getting Started Today

You now have everything you need to implement llms.txt and start improving your AI visibility. Here’s your action plan:

Today: Audit your 20-30 most important pages and draft your llms.txt structure

This Week: Create and upload your llms.txt file to your site root

This Month: Create markdown versions of your top 10 pages and monitor AI citations

This Quarter: Implement broader GEO strategies including structured data and content optimization

Ongoing: Review and update your llms.txt quarterly as you publish new content

The shift from traditional search to AI-powered discovery is already underway. Traditional search engine volume is predicted to drop 25% by 2026 and 50% by 2028, replaced by traffic from generative engines.

The question isn’t whether to adapt—it’s whether you’ll be among the early adopters who establish authority now, or among the laggards scrambling to catch up later.

Implement llms.txt today. Your future self will thank you when AI systems are citing your content instead of your competitors’.

---

Additional Resources

• Official llms.txt Specification: https://llmstxt.org/
• Mintlify Documentation: https://www.mintlify.com/docs/ai/llmstxt
• Answer.AI llms.txt Proposal: https://www.answer.ai/posts/2024-09-03-llmstxt.html
• GitBook llms.txt Guide: https://www.gitbook.com/blog/what-is-llms-txt

The future of search visibility is here. Are you ready? We made a LLMS.txt file here at CMC – see if you can find it!