The short answer
No. As of mid-2026, there is no credible evidence that publishing an llms.txt file gets you cited more often by AI search engines. Two independent studies of hundreds of thousands of real websites found that ChatGPT, Perplexity, Gemini, and Google almost never request the file, Google has publicly said its search systems do not use it, and no major AI company has committed to reading anyone else's llms.txt in production.
That is the honest answer, and we are writing it even though we ship an llms.txt generator ourselves. The file is cheap and harmless to publish, and it has one genuine use that has nothing to do with AI citations. But if someone is selling llms.txt to you as a way to win answers in ChatGPT or Perplexity, they are ahead of the evidence. Here is what the data actually shows, where the idea came from, and what really drives AI citations instead.
In a June 2026 study of 137,210 websites, 97 percent of llms.txt files received zero requests in a month, from any bot or human. Among the bots that did fetch one, most were not AI tools at all. Slackbot requested llms.txt files more often than Perplexity's crawler did.
What llms.txt actually is, and what it is not
llms.txt was proposed by Jeremy Howard, co-founder of Answer.AI, on September 3, 2024, with a specification published at llmstxt.org. The idea is reasonable on its face: large language models work with limited context windows, so instead of forcing a model to crawl and parse your entire site, you hand it a single curated markdown file that lists your most important pages with short descriptions.
The spec is deliberately simple. A valid llms.txt has:
- An H1 with your project or brand name.
- A blockquote summary, one short paragraph on what you do.
- H2 sections containing lists of markdown links, each with a brief note.
You will also see llms-full.txt in the wild. That is a companion file that concatenates your actual page content into one large markdown document, rather than just linking to it. Worth knowing: the full variant is an implementation convention popularized by tooling, not a core part of Howard's original spec.
The most important thing to understand is what llms.txt is not. It is a community proposal, not a ratified web standard, and not something any major AI vendor has agreed to support. That matters because the file is often bundled with a second one, ai.txt, and sold as a pair. The two are unrelated proposals, and the claims made for ai.txt are weaker still. Here is how the four files that get lumped together actually compare.
| File | What it is for | Who actually reads it | Status |
|---|---|---|---|
robots.txt | Tells crawlers which paths they may access | Every major crawler, including AI crawlers | Long-standing, universally honored |
sitemap.xml | Lists your indexable URLs for discovery | Google, Bing, and other search crawlers | Established standard |
llms.txt | A curated markdown index for language models | Mostly coding agents on docs sites. Answer engines almost never fetch it. | Community proposal, not vendor-adopted |
ai.txt | Loosely-defined content-usage and permissions signaling | No major AI vendor checks it | Separate proposal, not a standard |
Note the third column. robots.txt and sitemap.xml are read by the machines they were built for. llms.txt and ai.txt, for the most part, are not. That is the whole story in one table, and the rest of this article is the evidence behind it.
Who actually publishes llms.txt
Adoption is real, but it is niche and concentrated. The clearest number comes from Ahrefs, which checked 137,210 domains that received traffic in May 2026 and found roughly 28 percent had published an llms.txt file. That figure gets quoted as if 28 percent of the web has adopted it, which is wrong. Ahrefs Web Analytics skews heavily toward technically sophisticated sites, so the real web-wide number is far lower.
Look at who the adopters are and the pattern is obvious. The named names are companies like Anthropic, Stripe, Cursor, Mintlify, Zapier, Vercel, and Cloudflare. These are developer-tools and documentation companies, and many of them publish the file automatically because their docs platform generates it. That concentration is a clue about what llms.txt is genuinely good for, which we will come back to.
Do AI engines actually read it?
This is the question that matters, and it is where the evidence is both strongest and most one-sided.
What the vendors say
Google has said no, on the record and more than once. Gary Illyes stated at a Search Central event in 2025 that Google does not support llms.txt and is not planning to, and John Mueller has separately confirmed that no Google Search system reads or acts on it. When an llms.txt reference briefly appeared in Google's own developer documentation in December 2025, it was removed the same day, with Mueller clarifying that Search neither uses nor endorses it.
The other vendors have said nothing, which is its own kind of answer. OpenAI documents how to control its crawlers through robots.txt and makes no mention of llms.txt. Anthropic publishes an llms.txt on its own docs, but has not stated that Claude consumes third-party llms.txt files during inference. Perplexity and Microsoft have made no public commitment either. As of the first half of 2026, not one major AI company has publicly committed to reading or acting on someone else's llms.txt in production.
What the server logs show
Vendor statements are one thing. Server logs are harder to argue with, because they record what the bots actually did. Two independent log studies in 2026 reached the same conclusion from different angles.
| Study | Sample | Finding |
|---|---|---|
| Ahrefs (June 2026) | 137,210 domains, one month of request data | 97 percent of llms.txt files were never requested. Of the bots that did fetch one, 77 percent were not AI tools. Slackbot fetched llms.txt more often than PerplexityBot. AI bots never went looking for llms.txt files that did not exist. |
| Wislr (March 2026) | 288,566 CDN log files over 48 days, 12,099 bot requests | Zero requests for llms.txt or llm.txt from any AI bot. Meanwhile the same AI bots were actively crawling the actual page content (ChatGPT, Claude, Perplexity, and Google bots all showed up on real pages). |
Read those two rows together. The AI crawlers are busy. They are fetching real pages every day. They are simply not fetching the file that is supposed to make them understand your site. They go straight to the content, the same way they always have.
One honest nuance, because the goal here is accuracy and not a hit piece: Ahrefs did observe a small amount of llms.txt fetching, and the most active fetcher was Anthropic's Claude-Code, a coding agent, rather than an answer-and-search bot. So "literally zero" would be too strong. The accurate statement is that answer engines essentially ignore the file, and the meaningful fetching comes from coding agents on documentation sites. Hold that thought.
The comparison that stuck
In April 2025, John Mueller summed up the skeptic case on Reddit, and the line has been quoted ever since:
AFAIK none of the AI services have said they're using LLMs.TXT (and you can tell when you look at your server logs that they don't even check for it).
He then made the comparison that gave the debate its shape:
To me, it's comparable to the keywords meta tag. This is what a site-owner claims their site is about. Is the site really like that? Well, you can check it. At that point, why not just check the site directly?
That is the core problem. llms.txt asks an AI to trust your own description of your site, when the AI can simply read the site. For the citation use case, the file is redundant with content the engines are already crawling.
The honest counterargument
There are thoughtful people who defend llms.txt, and they are not wrong about everything. Two arguments are worth taking seriously.
The first is logical. Carolyn Shelby argued that the keywords-meta-tag comparison is unfair, because the old meta tag was a bag of keywords with nothing behind it, while llms.txt contains real URLs to real content that has to deliver when fetched. You cannot stuff it with lies the way people stuffed meta keywords. That is a fair point. It is also not evidence that llms.txt lifts citations. It only argues that the file is less gameable, which is a different claim.
The second is about a completely different use case, and this is the one that holds up. Documentation platforms like Mintlify report that coding agents and retrieval tools genuinely benefit from llms.txt: serving clean markdown instead of full HTML cuts the tokens an agent has to process, and a curated index helps agents find the right doc faster. This is the same signal the server logs showed, where the real fetching came from Claude-Code and similar tools. It lines up. The catch is that these are vendor reports about coding and documentation agents, not independent proof of answer-engine citations. Mintlify also sells llms.txt tooling, so weigh it accordingly.
Put the two lanes side by side and the picture is clear. There is a real, evidenced benefit for coding agents and documentation retrieval. There is no evidenced benefit for getting cited more in ChatGPT, Perplexity, Gemini, or Google's AI features. People conflate the two constantly.
Is there any proof it lifts citations?
We went looking for a controlled experiment, the kind that would settle this: take a set of comparable pages, add llms.txt to half, leave the other half alone, and measure whether the llms.txt group gets cited more. That study does not appear to exist. Every "llms.txt boosted our visibility" claim we could find was either anecdotal, or it actually tested something else.
The most common mix-up is with structured data. There are real experiments showing that adding schema markup, clean tables, and quotable answer formats can raise how often a page gets cited. Those results are about on-page structure. They are routinely cited as if they prove llms.txt works, when llms.txt is never the variable being tested. If you see a citation-lift number attached to llms.txt with no controlled comparison behind it, treat it as marketing.
| What people claim | What the evidence shows |
|---|---|
| AI engines read your llms.txt to understand your site | Logs of 100,000-plus sites show answer engines almost never request it |
| llms.txt gets you cited more in ChatGPT and Perplexity | No controlled study has shown this; the engines that cite do not fetch the file |
| Google uses llms.txt for AI Overviews | Google has said on the record that its search systems do not |
| ai.txt tells AI systems they are allowed to cite you | No major AI vendor checks ai.txt; it is not part of any adopted standard |
| You need both llms.txt and ai.txt for AI visibility | Neither has been shown to affect citations |
So should you make one?
Yes, with the right expectations. It costs almost nothing to publish, it does no harm, and it is sensible future-proofing in case a vendor adopts it later. The case is strongest if you run a documentation or API site that coding agents consume, because that is where the only real, measured benefit lives today. If you make one, follow the spec: an H1 with your name, a one-line summary, then short sections of links to your genuinely important pages, pointing at clean content. Keep it small and curated. The point is to fit a context window, not to dump your whole site.
What you should not do is publish llms.txt and believe you have done something for your AI search visibility. The things that actually get you cited are well documented, and none of them is a text file at your domain root:
- Be in the index. AI answers are built by retrieving from a search index and then synthesizing. If you are not indexed and crawlable, you cannot be cited. Server-rendered HTML matters here, because most AI crawlers do not run JavaScript.
- Get talked about elsewhere. Across multiple 2025 and 2026 studies, brand mentions across the web correlate with AI visibility far more strongly than backlinks do. Third-party coverage, reviews, and community discussion move the needle.
- Write content that is easy to quote. Direct answers near the top, clean tables, and clear structure get extracted and cited more readily than long narrative.
- Stay fresh and specific. AI-cited pages skew newer than typically-ranked pages, and original data gets quoted more than recycled opinion.
If you want the longer version of that list, see our guides on generative engine optimization and getting cited in Google AI Overviews. And if you want to see which domains the engines are actually citing right now, that is exactly what we track on our AI citation data pages.
How we think about it at RankBull
We build an llms.txt generator, and we are not going to pretend the file is magic to sell it. We position it honestly: it is a clean, machine-readable and human-readable map of your key content, it is genuinely useful if coding agents or documentation tools consume your site, and it is cheap insurance if the major engines change their minds. We ship one for our own site for those reasons.
What we will not do is tell you it boosts your ChatGPT citations, because the evidence says it does not. Our AI visibility tools and our monitoring focus on the things that actually move citations: index presence, brand mentions, content structure, and tracking where you already show up across ChatGPT, Claude, Perplexity, Gemini, and Google. Being the company willing to tell you a popular tactic does not work is, we think, a better long-term bet than being one more voice in the hype.
Sources and further reading
- llmstxt.org - the original specification, Jeremy Howard (Answer.AI), September 2024.
- Ahrefs: llms.txt server-log study - 137,210 domains, June 2026 (the 97 percent figure).
- Ahrefs: what is llms.txt - adoption rate and analysis, updated June 2026.
- Wislr: AI bot behavior log analysis - 288,566 log files, March 2026 (zero AI-bot requests).
- Search Engine Journal: Mueller's keywords-meta-tag comparison - April 2025.
- Search Engine Land: the counterargument - Carolyn Shelby, July 2025.
Frequently Asked Questions
Does llms.txt help you get cited by AI?
There is no credible evidence that it does. As of 2026, independent studies of hundreds of thousands of websites show that AI answer engines almost never request the file, and Google has publicly stated its search systems do not use it. No major AI vendor has committed to reading third-party llms.txt files in production.
Do ChatGPT, Perplexity, or Google read llms.txt?
Server-log studies show they almost never fetch it. In one June 2026 analysis of 137,210 domains, 97 percent of llms.txt files received zero requests in a month, and most of the fetches that did happen came from non-AI bots. The AI crawlers go straight to your actual page content instead.
Is llms.txt worth creating in 2026?
It is cheap and harmless, so there is little downside to publishing one. It is most worthwhile if you run a documentation or API site that coding agents like Cursor or Claude Code consume, because that is the one place it has a measured benefit. Do not expect it to increase your AI search citations.
What is the difference between llms.txt and robots.txt?
robots.txt controls which parts of your site crawlers may access, and every major crawler honors it. llms.txt is a proposed markdown index of your key content for language models, but no major AI engine has committed to using it, and server logs show they rarely fetch it.