There is a file called llms.txt at the root of this website. You can go and look at it. It lists what we do, who we work with and where the useful pages are, written in plain Markdown for a machine to read.
We also publish an article whose FAQ answers “Do I need an llms.txt file?” with “Probably not.”
Both of those are still true, and the gap between them is the honest version of where this idea has landed. So rather than argue it from first principles, here is what the measurements say.
What the server logs show
The most useful study on this is Ahrefs’ analysis of server requests across the 137,210 domains in its Web Analytics that received traffic in May 2026.
97 per cent of valid llms.txt files received no requests at all. Not from AI crawlers, not from anything.
The second finding matters more than the first, and it gets quoted far less often. No AI bot ever requested an llms.txt file that did not exist. Nothing was out there speculatively checking. Every request that landed did so because something already pointed at the file.
That is the whole argument in one sentence. A file nobody looks for is not a discovery mechanism, whatever else it might be.
The part that keeps this honest
It is not zero, and any article telling you it is zero is a year out of date.
Of the requests that did reach an llms.txt file in that dataset, roughly a fifth came from named AI agents. GPTBot accounted for about 4.5 per cent of all requests to those files, an indexer called statespace-indexer for about 3.5 per cent, ClaudeBot and OAI-SearchBot for well under one per cent each. PerplexityBot, ChatGPT-User and Claude-User all appear.
The most interesting name on the list is Claude-Code, which fetched llms.txt files more than any of the retrieval bots or consumer assistants. That is a coding agent, working through a documentation site because a developer told it to, not a search crawler building an index.
So the accurate present-tense statement is narrower than the popular one. Things do fetch llms.txt. They fetch it when a link or a human instruction sends them there. No major consumer AI search product uses it as a retrieval or ranking input.
Google has now written it down
For most of this file’s short life the argument rested on the absence of any vendor saying they used it. That changed in May 2026, and hardened in June.
Google’s own AI optimisation guidance, last updated on 10 July 2026, now names the file directly under a heading reading “Mythbusting generative AI search: what you don’t need to do”:
LLMS.txt files and other “special” markup: You don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn’t use them.
And then, more bluntly, in the same passage:
It’s completely fine if you decide to create and maintain LLMS.txt files (or other similar files) for other services or systems that use these files. Doing so will neither harm nor help your site’s visibility or rankings in Google Search, as Google Search ignores them.
The page’s closing advice puts the file in named company: “Prioritize effective SEO strategies over AEO/GEO hacks: For Google Search, you can ignore tactics like chunking content, creating unnecessary AI text files (like llms.txt), or pursuing inauthentic mentions.”
There is an older set of quotes doing the rounds from Google’s John Mueller, and they should be dated rather than presented as current. In June 2025 he wrote that consumer chatbots “will fetch your pages, for training and grounding, but none of them fetch the llms.txt file”, adding “Maybe they will tomorrow? Maybe I’ll win in the lottery tomorrow?” He scoped that himself to consumer chatbots, and the Ahrefs data shows agentic tooling has since moved. The documented Google position is the better citation, because it is Google’s rather than one person’s, and it is maintained.
Mueller did leave behind the most useful thing in the whole debate, which is a test you can run yourself. Check whether an AI system that already sends you meaningful traffic requests the file as often as it requests your robots.txt. If it does, the question is settled for your site. If it does not, you have your answer without needing anyone’s opinion.
Publishing one is not the same as reading one
This is where most of the confusion lives, and it is worth separating carefully.
OpenAI publishes an llms.txt, and so do Anthropic and Perplexity. All three were live when we checked on 19 August 2026, and the proposal’s own website lists them as evidence that the format has been adopted.
None of the three says anywhere in its crawler documentation that it reads anyone else’s.
We checked the actual documentation rather than the summaries. OpenAI’s bot page describes four user agents and exactly one control mechanism, which is robots.txt. Anthropic’s crawler article contains zero mentions of llms.txt. Perplexity’s crawler page mentions it twelve times, and every single one is documentation-platform boilerplate pointing at Perplexity’s own file, in phrasing repeated word for word on OpenAI’s pages. That looks like a shared documentation toolchain generating the file by default rather than three separate product decisions.
Google, the one company that has written a formal position, publishes no llms.txt on its own properties at all. We checked six of them. They return 404.
An AI lab shipping an index of its own developer docs tells you that hosted documentation platforms generate one automatically. It tells you nothing about what that lab’s crawler does when it visits your site.
What the vendors say does work
The affirmative guidance is consistent, and unglamorous.
Google’s answer is content people find useful and a site a crawler can actually read. Its guide is direct about the framing: “From Google Search’s perspective, optimizing for generative AI search is optimizing for the search experience, and thus still SEO.” Its single strongest line is that “Creating content that people find unique, compelling, and useful will likely influence your website’s presence in generative AI search in the long run more than any of the other suggestions in this guide.”
OpenAI’s only affirmative visibility instruction is a robots.txt allow. Sites opted out of OAI-SearchBot do not appear in ChatGPT search answers, so the recommended action is to permit that crawler and its published IP ranges. That is access, not a content file.
For agents specifically, Google’s own guidance names three ways a machine perceives your site: screenshots, raw HTML and the accessibility tree. A separate text file is not among them. Mueller’s summary of this in July 2026 is worth keeping: “If you’re trying to fix accessibility issues by making a separate agent-friendly version, you are just building technical debt. You’ll have to redo it multiple times. Just fix it.”
Anthropic is the notable gap. Its site-owner documentation is written entirely around transparency and blocking, covering Crawl-delay, Disallow and per-bot opt-out. There is no Anthropic equivalent of Google’s optimisation guide. If an agency tells you what to do to get cited by Claude, they are not quoting Anthropic, because Anthropic has not published an answer.
So why do we still have one
Three reasons, and none of them is traffic.
It costs nothing. It is a text file that took an hour to write and needs revisiting when the site changes shape.
It is a useful discipline. Writing a plain summary of what a business does, in a form with no room for design or copywriting to carry the weight, exposes waffle quickly. Ours got shorter twice.
And the downside is documented as zero. Google’s wording is that it will “neither harm nor help”, which is a rare thing to have in writing.
What it is not is a visibility strategy, and we would not put it on an invoice.
The cost nobody mentions
A file that describes your own site goes stale, and it goes stale silently because nothing is checking it.
While writing this piece we opened ours and found it claiming the site had 39 articles. It had 40. Nobody would ever have noticed, because the only reader who would care is a machine that, on this evidence, was not visiting.
That is the real running cost of a self-describing file. Not the hour to write it. The fact that it is a second copy of the truth, drifting quietly away from the first, with no test and no alert behind it. If you keep one, put it in whatever process already catches your stale content, or accept that it will end up describing a version of your business that no longer exists.
The short version
Publish one if you want to. It is cheap, Google has confirmed in writing that it cannot hurt you, and coding agents do fetch these files when a developer points them at one.
Do not publish one expecting to be found. Nothing goes looking for it. Ninety-seven per cent of them sat unread for a month across 137,000 domains, and no crawler in that dataset ever checked whether one existed.
The work that decides whether an AI system can use your site is the same work it always was. Pages a crawler can reach, content worth quoting, and a robots.txt that lets the right bots in.
Frequently asked questions
Does llms.txt actually do anything?
On current evidence, very little. Ahrefs’ analysis of 137,210 domains found 97 per cent of llms.txt files received no requests during May 2026, and no AI bot ever requested one that did not exist. Requests that did arrive came mostly from coding agents following a link, led by Claude-Code, rather than from search crawlers.
Does Google use llms.txt?
No, and it says so in its own documentation. Google’s AI optimisation guidance, updated 10 July 2026, states that Google Search “doesn’t use them” and that maintaining one “will neither harm nor help your site’s visibility or rankings in Google Search, as Google Search ignores them”.
OpenAI and Anthropic both publish an llms.txt. Doesn’t that prove it works?
It proves their documentation platforms generate one. Neither company’s crawler documentation says it reads llms.txt files on other people’s sites, and the references to the file on OpenAI’s and Perplexity’s crawler pages are identical boilerplate pointing at their own index. Publishing a file and reading yours are separate things.
Should I delete the llms.txt I already have?
There is no reason to. Google has stated in writing that it neither helps nor harms. The only real cost is that it drifts out of date without anyone noticing, so either fold it into whatever already reviews your content or expect it to go stale.
What should I do instead to appear in AI answers?
Make sure the right crawlers are allowed in your robots.txt, since OpenAI states that sites opting out of OAI-SearchBot will not appear in ChatGPT search answers. Serve content in HTML a crawler can read without executing JavaScript. Then write things worth citing. We covered the technical side in why your website is invisible to ChatGPT.
Flux Dynamics is a fractional CTO who builds. We check claims like these against server logs and vendor documentation before acting on them, because most AI visibility advice is repeated rather than measured. Tell us what you are considering.