Does llms.txt do anything?

The idea is reasonable. Give models a clean, plain-text map of your site so they don't have to reconstruct it from navigation and markup. Webflow added an upload field for it. Most AEO checklists list it near the top.

The numbers

Ahrefs looked at 137,210 domains receiving traffic in May 2026. Two findings, and the second is the telling one:

• 28% publish an llms.txt. Adoption is real and fast.
• 97% of those files received zero fetches in the month measured.

Zero is absolute. The file is being written and left unread.

Google's John Mueller has been about as direct as a search advocate gets, saying no AI system currently uses llms.txt and describing it as a temporary crutch for coding agents. No major vendor has documented reading it, and that covers OpenAI, Anthropic, Perplexity and Google.

Meanwhile robots.txt, sitemap.xml and the HTML of your pages are demonstrably fetched by all of them, constantly.

So why does this site have one

Three reasons, all of them small.

It costs nothing. Mine is a few hundred bytes, generated from the same list of pages as the sitemap. It cannot fall out of date independently because there's one source.

It's a cheap option on a standard that might land. If a vendor starts reading these files, the file is already there. If none ever does, I've spent a few hundred bytes.

Writing it is a useful exercise. Compressing a site to one plain-text page forces you to say what each page is for in one line. Every time I've done this I've found a page I couldn't describe, which is always a page that should not exist.

With clients the rule is simple. It is unbilled, it comes with no citation promise, and in a proposal it ranks below the things that are measurably read.

The measurement trap

Checking this yourself is where it gets hard. On Webflow, server logs come with Enterprise. So you cannot see whether anything fetched your llms.txt, your sitemap or your pages, by user agent, at all.

The workaround is to put the site behind Cloudflare and read the edge logs. Then you can answer the only question worth asking: did anything ask for this file, and which bot was it?

Until you can see that, every claim about what AI crawlers do on your site is somebody else's data.

What earns the fetch

Ordered by how much of my time each is worth:

• Content in the served HTML. Nothing downstream matters if the words are not in the response.
• One URL per idea. A thing that can be cited needs an address.
• robots.txt that does not block what you want read. I still find sites blocking the crawlers they're paying someone to court.
• A sitemap that matches reality. The most-cited study in this space found roughly a third of AI crawler fetches hitting 404s and could not explain why. Nobody has followed that up in twenty months, which tells you where the unclaimed ground is.
• llms.txt. Last, cheap, honest about its odds.

Ship the file. Rank it where the evidence puts it: last.