Why AI shopping agents can't buy from most stores yet

A page reaches a program as HTML, the markup the server sends. A person sees it after the browser has run the page's scripts and drawn the result. A program that does not run those scripts has only the HTML, so a price, a stock level or a list of sizes that a script fills in after loading is missing for it.

<title>Ridge trail shoe | Example store</title><h1>Ridge trail shoe</h1><img src="/img/ridge.jpg" alt="Ridge trail shoe, side view"><p class="price"><span id="price"></span></p><p class="stock"><span id="stock"></span></p><label>Size  <select name="size">    <option>40</option><option>41</option><option>42</option><option>43</option>    <option>44</option><option>45</option><option>46</option>  </select></label><script src="/app.js"></script>

The llms.txt proposal describes the problem from the agent's side: "An HTML page wraps its information in navigation, ads, and JavaScript, and converting it back into clean text is difficult and imprecise." It suggests that a site add a /llms.txt file with brief background, guidance and links to detailed Markdown files, and a clean Markdown version of each page at the same address with .md appended. Markdown is plain text with light marks for headings, lists and links. The proposal is titled "The /llms.txt file, v2", was last modified on 10 August 2026, and has one author, Jeremy Howard.

Another route is to ask for Markdown in the request, through the Accept header, the line where a client says which formats it wants. Cloudflare offers this on its own network as Markdown for Agents: on sites where the feature is turned on, a request whose Accept header includes text/markdown gets the store's HTML converted to Markdown by Cloudflare before the answer goes out.

The series covers what an agent can and cannot read on a product page, from raw HTML to structured data and Markdown versions, in How an AI agent reads a web page.

Before a store answers a request, its server, or a bot-protection service in front of it, may decide whether to let the visitor in. Software names itself in the User-Agent header, a line in each request that carries the client's name and version, and nothing stops a client from writing someone else's name there. The IETF draft on identifying automated traffic puts it this way: "User-Agent, when used alone, can be spoofed meaning anyone may attempt to act as that agent." It lists "IP allowlisting, User-Agent strings, or shared API keys" as the current methods and says they "have significant limitations".

The robots.txt file, where a site lists which crawlers may fetch which paths, does not settle the question. RFC 9309, the standard that defines it, describes rules "that crawlers are requested to honor" and says: "These rules are not a form of access authorization." The rules are matched against the name a crawler gives itself. For agents acting on a person's request, OpenAI's page on its bots says of ChatGPT-User: "Because these actions are initiated by a user, robots.txt rules may not apply."

A store that wants to let a shopper's agent through and keep scrapers out has weak signals to go on. In our reading, a protection rule that cannot tell the two apart has to let both in or turn both away.

The draft quoted above is Web Bot Auth. draft-ietf-webbotauth-httpsig-protocol-00, dated 1 September 2026, is an Internet-Draft of the IETF's webbotauth working group. In our words, an Internet-Draft is an early working document of the IETF, the body that publishes internet standards, and it can still change; the IETF's own boilerplate says such drafts may be cited only as "work in progress". This one is at revision 00, its first version, which is the -00 at the end of its name. Its aim is "to allow automated HTTP clients to cryptographically sign outbound requests, allowing HTTP servers to verify their identity with confidence." It builds on RFC 9421, HTTP Message Signatures. A signed request "MUST carry the Signature-Agent header", which says where the keys are, and the draft defines a key directory at /.well-known/http-message-signatures-directory. The agent's operator publishes a public key, the agent signs each request with the matching private key, and the store's side checks the signature.

How user agents, robots.txt and signed requests work, and what each one proves, is the subject of Bots, crawlers and agents: who is knocking.

An agent that can read the pages still has to learn what the store supports beyond them: whether it has an interface built for agents, which version that interface speaks, and how the store takes payment.

UCP answers this with a file at a known address. A well-known path is a fixed location under /.well-known/ on a domain, where software can look without being told. UCP's overview says: "Businesses publish their profile at /.well-known/ucp." The profile lists the store's services, capabilities and payment handlers, and "The services and payment_handlers registries MUST be present even when empty". UCP explained goes through the profile field by field.

ACP has a discovery file only as a proposal. /.well-known/acp.json is defined in a discovery RFC marked "Status: Proposal" and "Version: unreleased". The RFC says the file would let agents "determine whether a seller supports ACP" and learn its version, transports and capabilities, "all before creating a checkout session". It is not part of the released 2026-04-17 specification, so ACP gives an agent no released file to check. What the released ACP specification does define is in ACP explained.

Other well-known documents, HTTP Link headers and API catalogs are covered in How agents discover what a store offers.

A store can also offer agents tools to call, through MCP or WebMCP, or an agent of its own that other agents talk to over A2A. MCP, A2A and WebMCP for stores covers what each of the three defines, where it runs and what status it gives itself.

A checkout made for people runs over several pages: an account or guest form, an address, a delivery option, a payment page. Some steps are there to stop software. A CAPTCHA, a small puzzle meant to tell a person from a program, is one. An agent that fills in the pages as a person would has to get past each of them.

UCP and ACP both define a second way in, a checkout the store implements as an API, meaning a set of requests the store's software answers. UCP's checkout capability has five operations: Create, Get, Update, Complete and Cancel. ACP's Agentic Checkout API has endpoints to create, update, retrieve, complete and cancel a checkout session.

Both plan for cases the agent cannot finish alone. In UCP, the status requires_escalation means the checkout "requires information that cannot be provided via API, or buyer input is required", and "Businesses MUST provide continue_url" with it, an address where the buyer takes over in the store's own interface. ACP's draft Agentic Checkout RFC says the server "MUST set session.status to authentication_required when authentication (e.g., 3DS) is required". 3-D Secure is the card issuer's extra check, such as a code sent to the cardholder's phone.

Signing in is a separate problem. If the agent needs the shopper's account at the store, it needs permission to act on that account without holding the password. OAuth 2.0 is a way to grant that kind of limited access. Two IETF standards tell software where to find the details: RFC 8414 defines a metadata document at /.well-known/oauth-authorization-server that a client "can use to obtain the information needed to interact with an OAuth 2.0 authorization server", and RFC 9728 defines one for the protected resource at /.well-known/oauth-protected-resource. UCP lists an Identity Linking capability among its common capabilities.

What challenges, required sign-in and 3-D Secure ask of an agent, and what OAuth metadata tells it, is the subject of Why agents stall at checkout.

A card form expects the cardholder to type the card in, and the agent is not the cardholder. ACP and Stripe give the agent a token instead: a stand-in value the payment provider accepts in place of the card details, with limits attached.

ACP's Delegate Payment API "Tokenizes a credential for controlled usage by the merchant's PSP per the Allowance constraints". The PSP, or payment service provider, is the company that processes the store's payments. The allowance sets a maximum amount, a currency, an expiry, the checkout session and the merchant the token is for, and "Exactly one credential type is currently supported: card". The draft Payment Handlers RFC names the caller: "Agent calls delegate_payment with the Seller's PSP". Stripe's Shared Payment Tokens "grant sellers scoped access to customers' payment methods for agent-initiated purchases"; Stripe offers them as a preview.

The Agent Payments Protocol (AP2), which Google donated to the FIDO Alliance, uses mandates, which we read as signed records of what the buyer agreed to. Its specification, titled "Agentic Payment Protocol (v0.2)", defines a Checkout Mandate and a Payment Mandate. It describes a flow with the human present, where the user approves the final checkout and payment, and one without, where the user approves constraints and the agent signs within them. UCP's checkout capability refers to it: "The checkout has to be finalized manually by the user through a trusted UI unless the AP2 Mandates extension is supported."

In UCP and ACP the store stays the seller. UCP says "The business remains the Merchant of Record (MoR)", and agenticcommerce.dev says businesses "maintain their customer relationships as the merchant of record". The merchant of record is the party that sells to the buyer and answers for the sale. The two payment schemes address different parties: ACP's token is for "the merchant's PSP", and AP2's Payment Mandate goes to the credential provider, the networks and the merchant's payment processor.

Stripe's and ACP's tokens, what Adyen and PayPal document, Visa's Trusted Agent Protocol and AP2's mandates get their own article, How an agent pays without seeing your card. How the protocols divide these layers between them is set out in UCP vs ACP vs MCP vs AP2.

Some of this you can check yourself or ask your suppliers:

• Open a product page with the browser's "view source" and look for the price and stock level in the HTML. If they are missing, a program that does not run your scripts does not see them.
• Treat your robots.txt as a request to crawlers. RFC 9309 says its rules are not access authorization.
• Ask whoever runs your bot protection whether it can verify Web Bot Auth signatures, and how it treats agents that do not sign.
• Open https://your-domain/.well-known/ucp. A JSON document with a ucp member means something already publishes a UCP profile for your store.
• Ask your platform, or whoever maintains your checkout, whether it implements the UCP or ACP checkout and what it does when a checkout needs the buyer.
• Ask your payment provider whether it accepts ACP delegated tokens or Stripe Shared Payment Tokens.

You can also answer these checks below, one question per wall. Each answer says what it means for an agent and which article to read next.

Several of the documents behind these walls are early or not yet settled:

• Web Bot Auth is a working-group Internet-Draft at revision 00.
• llms.txt is a proposal, and Markdown for Agents is one company's feature.
• ACP calls itself beta, and its discovery file is a proposal.
• Stripe's Shared Payment Tokens are a preview.
• AP2 is at version 0.2, and its mandate types changed from version 0.1.
• UCP refers to a "UCP governing body" without naming it.