How ChatGPT, Perplexity and Google shop: what they read from your store

Both, and it differs by assistant. A feed is a file of your products, with prices and stock, that you send to the assistant's maker; OpenAI and Google both take one, and Microsoft uses its Merchant Center feeds. A crawler is a bot that visits your pages and reads them itself, the way a search engine does. Either way, this is the one corner of retail where you hope someone copies your price list. On top of that sits a third kind of visit: a fetch that happens because a person asked the assistant a question just now, which the vendors treat differently from a crawl. In our reading, a store that wants to be recommended needs to be readable to all three.

Here is what each maker says, in its own words:

A feed if you have one, and your pages if you let OAI-SearchBot in. OpenAI's commerce docs say product feeds "give ChatGPT the catalog data it needs to index your products", and that "Onboarding product feeds in ChatGPT is currently available to approved partners." Each product in the feed carries flags: is_eligible_search, where "false disables it and checkout eligibility", and the spec adds that "Checkout requires a separately enabled integration."

Without a feed, ChatGPT's search reads the web. OpenAI says "OAI-SearchBot is used to surface websites in search results in ChatGPT's search features", and is blunt about the opt-out: "Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers." Its other two bots do different jobs. "GPTBot is used to make our generative AI foundation models more useful and safe", and ChatGPT-User visits a page "When users ask ChatGPT or a CustomGPT a question". OpenAI also says its robots.txt settings are independent of one another, so a store can let OAI-SearchBot in while keeping GPTBot out.

Its own index, built by PerplexityBot, which Perplexity says "is designed to surface and link websites in search results on Perplexity. It is not used to crawl content for AI foundation models." Perplexity asks for it plainly: "To ensure your site appears in search results, we recommend allowing PerplexityBot in your site's robots.txt file". When a user asks a question, Perplexity-User "might visit a web page to help provide an accurate answer and include a link to the page in its response", and of that fetcher Perplexity says "Since a user requested the fetch, this fetcher generally ignores robots.txt rules."

For buying, Perplexity's help centre says Instant Buy is "currently available to all users in the United States", and only "with merchants who are compatible with our agentic checkout experience". Its pages we read do not describe a product feed for merchants.

Merchant Center and, in our reading, what Googlebot crawls for Search. Google calls Merchant Center "a free tool that helps millions of shoppers on Google discover, explore, and buy your products", and for a Buy button in AI Mode, "Only product listings using the native_commerce(checkout_eligibility) product attribute will display the "Buy" button." Google says that checkout is in an early access program, "available for select merchants at this time", for products eligible in the United States, Canada and Australia.

On the crawling side, Google is careful to separate Search from Gemini. Preferences addressed to Googlebot "affect Google Search (including Discover and all Google Search features)". Google-Extended is a robots.txt token with no crawler of its own, used to manage whether content may be used for training Gemini models and for grounding, and "Google-Extended does not impact a site's inclusion in Google Search nor is it used as a ranking signal in Google Search." Google-Agent, which visits pages on a user's request, is one of the user-triggered fetchers that "generally ignore robots.txt rules".

Microsoft points merchants to its own feed: "Although MMC is not required to sell through Copilot Checkout, product feeds from MMC help inform organic Copilot results." MMC is Microsoft Merchant Center. We found no Copilot crawler named on Microsoft's pages that we could read, so we do not name one.

Amazon names three bots. Amzn-SearchBot "is used to improve search experiences in Amazon products and services", and allowing it makes your content "eligible to appear in search experiences such as Alexa". Amzn-User "supports user actions, such as responding to Alexa queries", and "may not follow all robots.txt directives." Amazonbot "may be used to train Amazon AI models." Buy for Me, Amazon's beta for a subset of US customers, buys from other brands' sites by filling in the brand's own checkout; Amazon does not say which User-Agent it uses.

Anthropic has no shopping product of its own in the pages we read. Its bots are ClaudeBot for training, Claude-SearchBot, which "navigates the web to improve search result quality for users", and Claude-User for questions people ask. Anthropic says disabling Claude-User "may reduce your site's visibility for user-directed web search". Claude in Chrome, a browser extension that can "read, click, and navigate websites alongside you", visits your shop the way its user would.

Each kind of bot answers to it differently. Blocking a training crawler asks its vendor to leave your pages out of training: OpenAI says disallowing GPTBot indicates "that crawled content should not be used for training", and Anthropic says a ClaudeBot block "signals that the site's future materials should be excluded". Blocking a search crawler can take you out of that assistant's answers: OpenAI says opted-out sites "will not be shown", and Anthropic that it "may reduce your site's visibility". Blocking a user fetcher may do little, since four vendors say those fetchers may not follow robots.txt. Tick the bots your file turns away:

User-agent: GPTBot
User-agent: Amazonbot
User-agent: ClaudeBot
User-agent: Google-Extended
Disallow: /

User-agent: *
Allow: /

Two notes on reading it. The quotes are each vendor's claim about its own bots, which you can check in your logs but cannot otherwise verify; the AI agent traffic guide shows how. And our own robots.txt check, DISC-001, sorts the bots the same way. The "Block training only" preset gets a warning from it, because search and shopping agents can still read the store. A file that fully blocks a search crawler or a user fetcher fails. The robots.txt guide has a generator for the file itself.

That depends most on your platform. Shopify connects its stores to several assistants itself: its help centre says "Your products are made available to AI channels through Shopify Catalog" or, for Google, through the Google & YouTube channel or Merchant Center. It names ChatGPT, Google AI Mode and Gemini, Microsoft Copilot and Meta, and says agentic storefronts "is active by default for eligible stores". Anyone else applies channel by channel.

Where a vendor names a feed, the feed is what it reads; where it names a crawler, it reads your pages. A feed with prices that disagree with your pages gives an assistant two stories, and the prices, stock and variants guide covers keeping them in line.

This is our order, from the vendors' statements above:

1. Let the search crawlers in: OAI-SearchBot, PerplexityBot, Googlebot, Claude-SearchBot and Amzn-SearchBot. Each vendor ties its search results to its crawler.
2. Decide on the training crawlers separately. OpenAI and Amazon say each bot's setting is independent, Google says Google-Extended does not affect Search, and Anthropic lists a separate effect for each of its bots, so GPTBot, Amazonbot, Google-Extended and ClaudeBot can be blocked on their own.
3. Send feeds where you can: Merchant Center for Google, Microsoft Merchant Center for Copilot, and an application for ChatGPT's product feeds if you are eligible. On Shopify, check what Shopify Catalog already does for you.
4. Make your pages worth reading: product structured data with price and stock, and a checkout an agent can get through, which why agents stall at checkout covers.

How does ChatGPT find products?

From product feeds, which "give ChatGPT the catalog data it needs to index your products" and are "currently available to approved partners", and from the web through OAI-SearchBot. On Shopify, products reach ChatGPT through Shopify Catalog.

Does blocking GPTBot remove my store from ChatGPT?

Not from ChatGPT search, by OpenAI's account: GPTBot is for training, OAI-SearchBot is for search, and OpenAI says the settings are independent. Blocking OAI-SearchBot is what keeps you out: "Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers."

Where does Google AI Mode get product information?

From Merchant Center and, in our reading, what Googlebot crawls for Search. A Buy button needs the native_commerce(checkout_eligibility) attribute in your feed, and Google says that checkout is in an early access program, "available for select merchants", for products eligible in the United States, Canada and Australia.

Do AI assistants follow robots.txt?

Their crawlers say they do. Their user fetchers are another matter: OpenAI says robots.txt "may not apply" to ChatGPT-User, Perplexity says Perplexity-User "generally ignores robots.txt rules", Google says the same of its user-triggered fetchers, and Amazon says Amzn-User "may not follow all robots.txt directives".

Does Claude shop for people?

Anthropic's pages we read describe no shopping product. Claude-User and Claude-SearchBot read pages for answers, and Claude in Chrome browses sites alongside its user. Our agent readiness checker tells you whether your store lets these agents in.

The sources below were last checked on the date shown.