Scanner Category
LLMs.txt Checker
The llms.txt file is how you tell AI systems about your site in their own language. We validate your file against the specification, check linked resources, and ensure your content is LLM-ready.
What It Does
While robots.txt controls access, llms.txt provides context. It's a Markdown file at your domain root that gives AI systems a structured overview of your site — what it does, what content is available, and which pages are most important.
GEO Lantern's llms.txt checker validates your file against the llms.txt specification. We check for required elements (H1 title, blockquote summary), validate the structure of your sections, and fetch every linked resource to verify accessibility.
LLMs.txt accounts for 10% of your overall AI readiness score. As AI search engines increasingly look for explicit signals, having a well-structured llms.txt gives your site an advantage in how AI systems understand and prioritise your content.
The Specification
LLMs.txt Structure
The llms.txt specification defines a clear structure that AI systems know how to parse.
H1 Title
The top-level heading that identifies your site or project. This is the first thing LLMs read when processing your llms.txt file.
Blockquote Summary
A brief description of your site immediately after the H1. This gives LLMs a quick summary of what your site is about.
H2 Sections
Topic-based sections that organise your content into logical groups. Each section can contain descriptive text and links to relevant pages.
Linked Resources
Links to key pages on your site that you want LLMs to know about. These help AI systems understand your site structure and important content.
Optional Section
A section marked "Optional" for supplementary content. LLMs can deprioritise this section when processing context-limited requests.
Validation
What We Validate
File Presence
We check for /llms.txt at your domain root, plus variants like /llms-full.txt, /llms.md, and /llms.json.
Spec Compliance
We validate the structure against the llms.txt specification — H1 title, blockquote summary, H2 sections, and proper link formatting.
Link Accessibility
We fetch each linked resource in your llms.txt to verify they are accessible and return valid content.
Token Count
We calculate the token count of your llms.txt content to ensure it fits within common LLM context windows.
Content Quality
We evaluate whether the content provides genuinely useful information that helps LLMs understand your site.
Step by Step
How It Works
Our llms.txt validation runs automatically during every scan.
Scan your URL
Enter your domain URL and GEO Lantern checks for llms.txt at the root of your site.
File detection
We look for /llms.txt, /llms-full.txt, and variant formats. If found, we fetch the content for validation.
Specification validation
The file structure is validated against the llms.txt spec — checking for required elements, proper formatting, and content organisation.
Resource checking
Every linked resource in your llms.txt is fetched to verify accessibility. Broken links are flagged for correction.
FAQ
Frequently Asked Questions
What is llms.txt?
llms.txt is an emerging standard that provides LLMs (Large Language Models) with a concise, structured overview of your website. Similar to how robots.txt communicates with crawlers, llms.txt communicates directly with AI systems. It is a Markdown file placed at the root of your domain (/llms.txt) that contains a title, summary, and links to your most important content.
How does llms.txt differ from robots.txt?
robots.txt controls access — it tells crawlers what they can and cannot fetch. llms.txt provides context — it tells AI systems what your site is about and what content is most important. They serve complementary purposes. A site can have both, and ideally should have both for maximum AI readiness.
Is llms.txt widely adopted?
llms.txt is an emerging standard gaining traction in the AI and web communities. While not yet universally supported by all AI systems, early adoption positions your site favourably as AI search engines increasingly look for explicit signals about content. GEO Lantern includes llms.txt as 10% of your AI readiness score.
What does GEO Lantern check in my llms.txt?
GEO Lantern validates: file presence at /llms.txt, specification compliance (proper H1, blockquote, H2 sections), accessibility of all linked resources, token count relative to common LLM context windows, and overall content quality. We also check for variant files like /llms-full.txt.
Can GEO Lantern help me create an llms.txt file?
Yes. GEO Lantern offers a free LLMs.txt Generator tool that guides you through creating a specification-compliant llms.txt file. You fill in your site details, sections, and links, and the tool generates the properly formatted Markdown content ready to deploy.
What should I include in my llms.txt?
Start with a clear title (H1) and a one-line summary (blockquote) of what your site does. Then add H2 sections for your main content areas, with links to your most important pages. Include an Optional section for supplementary content that AI systems can deprioritise when context is limited.
Ready to See Your Score?
Run a free AI readiness scan and discover exactly how AI search engines perceive your website.