# Does llms.txt work? What the evidence says

What the llms.txt proposal promises, what Google says about it, why Lighthouse looks for the file but shrugs when it is missing, and what is left for a store once each claim is checked.

By Valerii Checha, 15 Nov 2026

Take the llms.txt file off a site and run Chrome's Lighthouse on it, and the llms.txt audit does not fail. Its documentation says a missing file is marked "Not Applicable (N/A), as providing the file is optional at the moment." Lighthouse looks for the file and is perfectly relaxed when it isn't there.

So, does llms.txt work? For Google Search, no: Google says Search "ignores them" and that a file "will neither harm nor help". For AI assistants, nobody knows from first-party sources, because none of the vendors whose crawler pages we read says its bots read a site's llms.txt. The file is cheap, optional and harmless, and the evidence that it changes anything for a store is thin.

If you have already decided to publish a file, our [llms.txt guide](https://storewalk.ai/guides/llms-txt-for-online-stores) covers what a store's should hold, with a builder. This piece weighs the evidence instead, starting with the proposal's own claims.

## What does the llms.txt proposal promise?

The proposal is "The /llms.txt file, v2" by Jeremy Howard, published on 3 September 2024 and modified on 10 August 2026. It describes itself as "A proposal to standardise on using an /llms.txt file to provide information to help agents use a website." No standards body has adopted it.

Its promise is about agents doing less work. "Agents are expected to view or search llms.txt to find the information they need, then follow the relevant links." The format is light: "An H1 with the name of the project or site. This is the only required section", then an optional summary and lists of links.

The v2 text also reports on itself. After "two years of adoption", the author writes, "thousands of sites publish an llms.txt file, documentation platforms generate one automatically, and Chrome's Lighthouse audits sites for one". He says where it is used most: "software documentation, where coding agents follow them to find API references and tutorials." A shop appears only in passing, as "a business outlining its structure and policies". The file was designed with documentation sites in mind.

## What does Google say about llms.txt?

Google is unusually direct. Its guide to generative AI features, last updated on 10 July 2026, puts llms.txt first under "Here are a few things you can ignore for Google Search":

> You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them.

It goes on: "It's completely fine if you decide to create and maintain LLMS.txt files (or other similar files) for other services or systems that use these files. Doing so will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them." And Google heads off one hopeful reading: if Googlebot fetches the file, "this doesn't mean that the file is treated in a special way."

That covers AI Overviews and AI Mode, since both are Search. Our [GEO vs SEO article](https://storewalk.ai/blog/geo-vs-seo-for-online-stores) has the rest of what Google says those features need, which is mostly ordinary SEO.

## Why does Lighthouse check for llms.txt if Google Search ignores it?

Lighthouse is a Chrome developer tool, and its Agentic Browsing category tests how well a page works for agents. Google's own scoring page says the category is "experimental and based on proposed standards", needs Chrome 150 or later, and "does not have a weighted average score from 0 to 100". One of its audits "Checks for the presence of a machine-readable summary at the domain root."

The audit's own page is more modest than the people who quote it. It calls llms.txt "an emerging convention" and gives its reason in a "may": "Without this file, agents may spend more time crawling the site". What it penalises is a broken server: "Lighthouse flags the pages if a server error occurs when attempting to retrieve the llms.txt file." A 404 is N/A. So one part of Google ignores the file, another politely asks after it, and neither will mark you down for not having one. Big companies contain multitudes.

## Which llms.txt claims hold up?

Here is what a store owner tends to hear, each checked against the page that would settle it:

Six things a store hears about llms.txt, against what the sources say.

“llms.txt helps you rank in Google’s AI answers” Not supported. What the source says: Google lists the file under things “you can ignore for Google Search”: Search “itself doesn’t use them”, and a file “will neither harm nor help your site’s visibility or rankings in Google Search, as Google Search ignores them.” Our reading: For Google Search, including AI Overviews and AI Mode, the file does nothing either way.

Source: [Google Search Central: Optimizing for generative AI features](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide).

“Google’s Lighthouse checks for it” Partly. What the source says: Lighthouse’s Agentic Browsing category “Checks for the presence of a machine-readable summary at the domain root.” The category is “experimental and based on proposed standards” and needs Chrome 150 or later. Our reading: True, in an experimental category with no 0 to 100 score. Chrome’s tooling looks for the file while Google Search ignores it.

Source: [Chrome for Developers: Lighthouse agentic browsing scoring](https://developer.chrome.com/docs/lighthouse/agentic-browsing/scoring).

“Lighthouse marks you down without one” Not supported. What the source says: “If the file is not provided by the server (resulting in a 404), the audit is marked as Not Applicable (N/A), as providing the file is optional at the moment.” What it flags is “a server error”. Our reading: A missing file costs nothing in Lighthouse. A file that errors does.

Source: [Chrome for Developers: Lighthouse llms.txt audit](https://developer.chrome.com/docs/lighthouse/agentic-browsing/llms-txt).

“ChatGPT, Claude and Perplexity read it” No first-party evidence. What the source says: When we read the crawler pages of OpenAI, Anthropic, Perplexity, Google, Apple, Meta and Amazon on 3 November 2026, none said its bots read a site’s llms.txt, and none said they don’t. The labs do publish llms.txt files for their own documentation. Our reading: Unknown. Publishing a file for your own docs says nothing about reading anyone else’s.

Source: [Storewalk: llms.txt for online stores](https://storewalk.ai/guides/llms-txt-for-online-stores).

“Everyone has one now” Partly. What the source says: The proposal’s author writes that “thousands of sites publish an llms.txt file” and that “llms.txt files are used most heavily for software documentation, where coding agents follow them”. Our reading: Thousands is the author’s word, not a count we can check. The heavy use he describes is documentation, not shops.

Source: [The /llms.txt file, v2](https://llmstxt.org/).

“It saves agents a crawl” No first-party evidence. What the source says: Lighthouse’s audit page: “Without this file, agents may spend more time crawling the site”. The proposal: “Agents are expected to view or search llms.txt to find the information they need, then follow the relevant links.” Our reading: That is the design and a “may”. Neither page publishes a measurement of agents doing it.

Sources: [Chrome for Developers: Lighthouse llms.txt audit](https://developer.chrome.com/docs/lighthouse/agentic-browsing/llms-txt) and [The /llms.txt file, v2](https://llmstxt.org/).

Two claims hold in part, two fail outright and two have no first-party evidence either way. The one that most often goes unchecked is "everyone has one". The author's "thousands" is not a figure we can verify, and his own description puts most of them in software documentation.

## Do AI assistants read a store's llms.txt?

They don't say. Our llms.txt guide read the crawler and help pages of the main AI vendors on 3 November 2026:

Who says what about llms.txt, in their own words.

- Google Search: Says it ignores llms.txt and other AI text files, and that adding one neither helps nor harms. In its own words: “Doing so will neither harm nor help your site’s visibility or rankings in Google Search, as Google Search ignores them.” Source: [Google Search Central: AI optimization guide](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide).
- Chrome Lighthouse: Its Agentic Browsing category, labelled experimental, checks that an llms.txt is present. Presence is what it describes. In its own words: “Checks for the presence of a machine-readable summary at the domain root.” Source: [Chrome for Developers: Agentic Browsing scoring](https://developer.chrome.com/docs/lighthouse/agentic-browsing/scoring).
- OpenAI, Anthropic, Gemini: The spec says the labs publish llms.txt files for their own developer docs. That is publishing; their crawler pages we read do not say their bots read yours. In its own words: “The AI labs themselves publish llms.txt files for their own developer docs: OpenAI, Anthropic, and Gemini.” Source: [llmstxt.org: The /llms.txt file, v2](https://llmstxt.org/).
- Shopify: Writes a default for every store and serves it at /llms.txt as well as /agents.md. In its own words: “The paths /llms.txt and /llms-full.txt also point to this content by default.” Source: [Shopify.dev changelog: customize llms.txt, llms-full.txt and agents.md](https://shopify.dev/changelog/posts/customize-llmstxt-llms-fulltxt-and-agentsmd).
- Storewalk (ours): Our DISC-002 is a small, optional check: it passes a file with an H1, at least 20 lines and links that resolve. In its own words: “llms.txt present and non-trivial” Source: [Storewalk: DISC-002](https://storewalk.ai/checks/DISC-002).

We found no statement from an AI vendor that its assistants or crawlers read a store’s llms.txt. That is an absence in the pages we read on 3 November 2026, not a denial.

The labs publish llms.txt files for their own developer docs, and the proposal names OpenAI, Anthropic and Gemini as examples. That shows they find the format useful for documentation they write. It does not show their shopping or search bots fetch yours. We also leave out third-party log studies that claim to count bot visits to llms.txt files: we could not check their data, and none of them is the vendors' own word.

If a vendor publishes that its agents read llms.txt on product sites, this answer changes, and we will update this article.

## What happens when your server answers /llms.txt?

How your server answers matters more than whether the file exists. A missing file is harmless in Lighthouse; a server error is not. Pick the answer your store gives and, for a Markdown file, paste it in:

How your server answers /llms.txt, and what Lighthouse, Google and our scan make of it.

On the page, you pick what your server sends for /llms.txt, say whether bot management fronts the store, and paste the file when it is Markdown. What each reader makes of each answer (the Markdown case uses the sample file, which is short):

A Markdown file:

- Lighthouse’s llms.txt audit (our reading): Not flagged: its page names only a server error and a 404.
- Google Search: Ignores the file in every case: “Google Search ignores them.”
- Our DISC-002: WARN: llms.txt present but is shorter than 20 lines.

Your home page (200, HTML):

- Lighthouse’s llms.txt audit: Its page does not say what happens to a 200 that is an HTML page.
- Google Search: Ignores the file in every case: “Google Search ignores them.”
- Our DISC-002: FAIL: llms.txt missing.

Not found (404):

- Lighthouse’s llms.txt audit: “the audit is marked as Not Applicable (N/A)”.
- Google Search: Ignores the file in every case: “Google Search ignores them.”
- Our DISC-002: FAIL: llms.txt missing.

A server error (500):

- Lighthouse’s llms.txt audit: “Lighthouse flags the pages if a server error occurs”.
- Google Search: Ignores the file in every case: “Google Search ignores them.”
- Our DISC-002: FAIL: llms.txt missing.

A bot challenge page:

- Lighthouse’s llms.txt audit: Its page does not say what happens to a challenge page.
- Google Search: Ignores the file in every case: “Google Search ignores them.”
- Our DISC-002: SKIPPED: llms.txt could not be assessed, because bot management answered instead of the shop.

A server error on a store fronted by bot management: SKIPPED: the store is fronted by bot management, so its HTTP 500 is not decidable.

DISC-002 is a lightly weighted check (5 points, against 25 for our product structured data check and 15 for our no-JavaScript check). It checks only the first 5 links, and fetches only those on your own domain; links elsewhere count as loading. It needs at least 20 non-empty lines: our threshold, not the spec’s, which requires only the H1.

Sources: [Chrome for Developers: Lighthouse llms.txt audit](https://developer.chrome.com/docs/lighthouse/agentic-browsing/llms-txt), [Google Search Central: Optimizing for generative AI features](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide) and [Storewalk: DISC-002, llms.txt present and non-trivial](https://storewalk.ai/checks/DISC-002).

Our own scan is stricter than Lighthouse on one point. [DISC-002](https://storewalk.ai/checks/DISC-002) fails a missing file, and a home page served at `/llms.txt` as well, since an agent asking for the file gets HTML it cannot use. That happens on storefronts that answer every unknown path with a page. The check carries 5 points, against 25 for our product structured data check and 15 for our no-JavaScript check, which is roughly how much we think the file matters: something, not much. When bot protection answers instead of your store, DISC-002 reports SKIPPED rather than guessing.

## What would change the answer?

Three kinds of evidence would move llms.txt from "harmless" to "worth doing well" for a store:

- A first-party statement from an AI vendor that its shopping or search agents read a site's llms.txt.
- A measurement published by a vendor or a platform, saying what agents did differently on sites that have one.
- Lighthouse, or another tool people rely on, changing the 404 case from N/A to a failure.

We found none of the three. Until one turns up, our advice is to publish a short, correct file if you like, make sure the URL never errors, and spend the real effort on what agents demonstrably read: robots.txt, product markup and pages that work without JavaScript. Those sit with the rest of [what stops agents buying from a store](https://storewalk.ai/blog/why-agents-cant-buy), and if you [scan your store](https://storewalk.ai/), it grades them next to DISC-002, so you can see its 5 points in proportion.

## Frequently asked questions

### Does llms.txt help SEO?

Not in Google Search. Google says Search ignores llms.txt and that a file "will neither harm nor help your site's visibility or rankings in Google Search."

### Does Lighthouse require llms.txt?

No. Its experimental Agentic Browsing category checks for the file, but a missing file (a 404) is marked Not Applicable. Lighthouse flags only a server error when it fetches the file.

### Do ChatGPT, Claude or Perplexity use llms.txt?

None of their crawler pages that we read says their bots read a site's llms.txt, and none says they don't. They publish llms.txt files for their own developer documentation.

### Should an online store still publish llms.txt?

It costs little, and Google says having one is "completely fine". If you publish one, keep it short and accurate, and make sure `/llms.txt` returns the file, not your home page or an error. Our [llms.txt guide](https://storewalk.ai/guides/llms-txt-for-online-stores) has a builder.

### How does Storewalk check llms.txt?

[DISC-002](https://storewalk.ai/checks/DISC-002) fetches `/llms.txt` and passes a Markdown file with an H1, at least 20 non-empty lines and links that load. It is one of our most lightly weighted checks, and a scan runs it alongside the ones that carry more weight.

The sources below were last checked on the date shown.

## Sources

- [The /llms.txt file, v2](https://llmstxt.org/)
- [Google Search Central: Optimizing for generative AI features](https://developers.google.com/search/docs/fundamentals/ai-optimization-guide)
- [Chrome for Developers: Lighthouse agentic browsing scoring](https://developer.chrome.com/docs/lighthouse/agentic-browsing/scoring)
- [Chrome for Developers: Lighthouse llms.txt audit](https://developer.chrome.com/docs/lighthouse/agentic-browsing/llms-txt)

Last checked 4 Nov 2026
