``` On the page the price and stock lines are marked. Marked lines stay empty until /app.js runs. | On the page | Raw HTML | | ------------------------- | -------- | | Product name | Yes | | Price | Missing | | In stock | Missing | | Sizes | Yes | | Returns note in the photo | Missing | The [llms.txt proposal](https://llmstxt.org/) describes the problem from the agent's side: "An HTML page wraps its information in navigation, ads, and JavaScript, and converting it back into clean text is difficult and imprecise." It suggests that a site add a `/llms.txt` file with brief background, guidance and links to detailed Markdown files, and a clean Markdown version of each page at the same address with `.md` appended. Markdown is plain text with light marks for headings, lists and links. The proposal is titled "The /llms.txt file, v2", was last modified on 10 August 2026, and has one author, Jeremy Howard. Another route is to ask for Markdown in the request, through the `Accept` header, the line where a client says which formats it wants. Cloudflare offers this on its own network as [Markdown for Agents](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/): on sites where the feature is turned on, a request whose `Accept` header includes `text/markdown` gets the store's HTML converted to Markdown by Cloudflare before the answer goes out. The series covers what an agent can and cannot read on a product page, from raw HTML to structured data and Markdown versions, in [How an AI agent reads a web page](https://storewalk.ai/blog/how-agents-read-a-page). ## Proving who the agent is Before a store answers a request, its server, or a bot-protection service in front of it, may decide whether to let the visitor in. Software names itself in the `User-Agent` header, a line in each request that carries the client's name and version, and nothing stops a client from writing someone else's name there. The IETF draft on identifying automated traffic puts it this way: "User-Agent, when used alone, can be spoofed meaning anyone may attempt to act as that agent." It lists "IP allowlisting, User-Agent strings, or shared API keys" as the current methods and says they "have significant limitations". The robots.txt file, where a site lists which crawlers may fetch which paths, does not settle the question. [RFC 9309](https://www.rfc-editor.org/rfc/rfc9309), the standard that defines it, describes rules "that crawlers are requested to honor" and says: "These rules are not a form of access authorization." The rules are matched against the name a crawler gives itself. For agents acting on a person's request, OpenAI's [page on its bots](https://developers.openai.com/api/docs/bots) says of ChatGPT-User: "Because these actions are initiated by a user, robots.txt rules may not apply." A store that wants to let a shopper's agent through and keep scrapers out has weak signals to go on. In our reading, a protection rule that cannot tell the two apart has to let both in or turn both away. The draft quoted above is Web Bot Auth. [draft-ietf-webbotauth-httpsig-protocol-00](https://datatracker.ietf.org/doc/draft-ietf-webbotauth-httpsig-protocol/), dated 1 September 2026, is an Internet-Draft of the IETF's webbotauth working group. In our words, an Internet-Draft is an early working document of the IETF, the body that publishes internet standards, and it can still change; the IETF's own boilerplate says such drafts may be cited only as "work in progress". This one is at revision 00, its first version, which is the `-00` at the end of its name. Its aim is "to allow automated HTTP clients to cryptographically sign outbound requests, allowing HTTP servers to verify their identity with confidence." It builds on [RFC 9421](https://www.rfc-editor.org/rfc/rfc9421), HTTP Message Signatures. A signed request "MUST carry the Signature-Agent header", which says where the keys are, and the draft defines a key directory at `/.well-known/http-message-signatures-directory`. The agent's operator publishes a public key, the agent signs each request with the matching private key, and the store's side checks the signature. How user agents, robots.txt and signed requests work, and what each one proves, is the subject of [Bots, crawlers and agents: who is knocking](https://storewalk.ai/blog/bots-crawlers-agents). ## Finding what the store offers An agent that can read the pages still has to learn what the store supports beyond them: whether it has an interface built for agents, which version that interface speaks, and how the store takes payment. UCP answers this with a file at a known address. A well-known path is a fixed location under `/.well-known/` on a domain, where software can look without being told. UCP's [overview](https://ucp.dev/latest/specification/overview/) says: "Businesses publish their profile at `/.well-known/ucp`." The profile lists the store's services, capabilities and payment handlers, and "The services and payment_handlers registries MUST be present even when empty". [UCP explained](https://storewalk.ai/blog/ucp-explained) goes through the profile field by field. ACP has a discovery file only as a proposal. `/.well-known/acp.json` is defined in a [discovery RFC](https://github.com/agentic-commerce-protocol/agentic-commerce-protocol/blob/main/rfcs/rfc.discovery.md) marked "Status: Proposal" and "Version: unreleased". The RFC says the file would let agents "determine whether a seller supports ACP" and learn its version, transports and capabilities, "all before creating a checkout session". It is not part of the released `2026-04-17` specification, so ACP gives an agent no released file to check. What the released ACP specification does define is in [ACP explained](https://storewalk.ai/blog/acp-explained). Other well-known documents, HTTP Link headers and API catalogs are covered in [How agents discover what a store offers](https://storewalk.ai/blog/how-agents-discover-a-store). A store can also offer agents tools to call, through MCP or WebMCP, or an agent of its own that other agents talk to over A2A. [MCP, A2A and WebMCP for stores](https://storewalk.ai/blog/mcp-a2a-webmcp-for-stores) covers what each of the three defines, where it runs and what status it gives itself. ## Getting through checkout A checkout made for people runs over several pages: an account or guest form, an address, a delivery option, a payment page. Some steps are there to stop software. A CAPTCHA, a small puzzle meant to tell a person from a program, is one. An agent that fills in the pages as a person would has to get past each of them. UCP and ACP both define a second way in, a checkout the store implements as an API, meaning a set of requests the store's software answers. UCP's [checkout capability](https://ucp.dev/latest/specification/shopping/checkout/) has five operations: Create, Get, Update, Complete and Cancel. ACP's [Agentic Checkout API](https://github.com/agentic-commerce-protocol/agentic-commerce-protocol/blob/main/spec/2026-04-17/openapi/openapi.agentic_checkout.yaml) has endpoints to create, update, retrieve, complete and cancel a checkout session. Both plan for cases the agent cannot finish alone. In UCP, the status `requires_escalation` means the checkout "requires information that cannot be provided via API, or buyer input is required", and "Businesses MUST provide continue_url" with it, an address where the buyer takes over in the store's own interface. ACP's draft [Agentic Checkout RFC](https://github.com/agentic-commerce-protocol/agentic-commerce-protocol/blob/main/rfcs/rfc.agentic_checkout.md) says the server "MUST set `session.status` to `authentication_required` when authentication (e.g., 3DS) is required". 3-D Secure is the card issuer's extra check, such as a code sent to the cardholder's phone. Signing in is a separate problem. If the agent needs the shopper's account at the store, it needs permission to act on that account without holding the password. OAuth 2.0 is a way to grant that kind of limited access. Two IETF standards tell software where to find the details: [RFC 8414](https://www.rfc-editor.org/rfc/rfc8414) defines a metadata document at `/.well-known/oauth-authorization-server` that a client "can use to obtain the information needed to interact with an OAuth 2.0 authorization server", and [RFC 9728](https://www.rfc-editor.org/rfc/rfc9728) defines one for the protected resource at `/.well-known/oauth-protected-resource`. UCP lists an Identity Linking capability among its common capabilities. What challenges, required sign-in and 3-D Secure ask of an agent, and what OAuth metadata tells it, is the subject of [Why agents stall at checkout](https://storewalk.ai/blog/why-agents-stall-at-checkout). ## Paying on someone's behalf A card form expects the cardholder to type the card in, and the agent is not the cardholder. ACP and Stripe give the agent a token instead: a stand-in value the payment provider accepts in place of the card details, with limits attached. ACP's [Delegate Payment API](https://github.com/agentic-commerce-protocol/agentic-commerce-protocol/blob/main/spec/2026-04-17/openapi/openapi.delegate_payment.yaml) "Tokenizes a credential for controlled usage by the merchant's PSP per the Allowance constraints". The PSP, or payment service provider, is the company that processes the store's payments. The allowance sets a maximum amount, a currency, an expiry, the checkout session and the merchant the token is for, and "Exactly one credential type is currently supported: card". The draft [Payment Handlers RFC](https://github.com/agentic-commerce-protocol/agentic-commerce-protocol/blob/main/rfcs/rfc.payment_handlers.md) names the caller: "Agent calls `delegate_payment` with the Seller's PSP". Stripe's [Shared Payment Tokens](https://docs.stripe.com/agentic-commerce/concepts/shared-payment-tokens) "grant sellers scoped access to customers' payment methods for agent-initiated purchases"; Stripe offers them as a preview. The Agent Payments Protocol (AP2), which Google donated to the FIDO Alliance, uses mandates, which we read as signed records of what the buyer agreed to. Its [specification](https://ap2-protocol.org/ap2/specification/), titled "Agentic Payment Protocol (v0.2)", defines a Checkout Mandate and a Payment Mandate. It describes a flow with the human present, where the user approves the final checkout and payment, and one without, where the user approves constraints and the agent signs within them. UCP's checkout capability refers to it: "The checkout has to be finalized manually by the user through a trusted UI unless the AP2 Mandates extension is supported." In UCP and ACP the store stays the seller. UCP says "The business remains the Merchant of Record (MoR)", and [agenticcommerce.dev](https://www.agenticcommerce.dev/) says businesses "maintain their customer relationships as the merchant of record". The merchant of record is the party that sells to the buyer and answers for the sale. The two payment schemes address different parties: ACP's token is for "the merchant's PSP", and AP2's Payment Mandate goes to the credential provider, the networks and the merchant's payment processor. Stripe's and ACP's tokens, what Adyen and PayPal document, Visa's Trusted Agent Protocol and AP2's mandates get their own article, [How an agent pays without seeing your card](https://storewalk.ai/blog/how-agents-pay). How the protocols divide these layers between them is set out in [UCP vs ACP vs MCP vs AP2](https://storewalk.ai/blog/ucp-vs-acp-vs-mcp-vs-ap2). ## What a store owner can do now, and what is still open Some of this you can check yourself or ask your suppliers: - Open a product page with the browser's "view source" and look for the price and stock level in the HTML. If they are missing, a program that does not run your scripts does not see them. - Treat your robots.txt as a request to crawlers. RFC 9309 says its rules are not access authorization. - Ask whoever runs your bot protection whether it can verify Web Bot Auth signatures, and how it treats agents that do not sign. - Open `https://your-domain/.well-known/ucp`. A JSON document with a `ucp` member means something already publishes a UCP profile for your store. - Ask your platform, or whoever maintains your checkout, whether it implements the UCP or ACP checkout and what it does when a checkout needs the buyer. - Ask your payment provider whether it accepts ACP delegated tokens or Stripe Shared Payment Tokens. You can also answer these checks below, one question per wall. Each answer says what it means for an agent and which article to read next. Five questions about your store, one for each wall. On the page each question can be answered yes, no or not sure, and each answer says what it means for an agent. 1. Reading the page. Are the price and the stock level in your product page’s HTML? Open a product page, choose “view source” in your browser and search for the price. - Yes: Then a program that does not run your scripts can read them. Google notes that “not all bots can run JavaScript”. Source: [Google: JavaScript SEO basics](https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics). Read on: [How an AI agent reads a web page](https://storewalk.ai/blog/how-agents-read-a-page). - No: Then a script fills them in, and a program that does not run it never sees them. Google describes pages where “the initial HTML does not contain the actual content”. Source: [Google: JavaScript SEO basics](https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics). Read on: [How an AI agent reads a web page](https://storewalk.ai/blog/how-agents-read-a-page). - Not sure: Then find out first; choose Yes or No to see what each case means for an agent. Read on: [How an AI agent reads a web page](https://storewalk.ai/blog/how-agents-read-a-page). 2. Proving who the agent is. Can your bot protection verify Web Bot Auth signatures? Ask whoever runs your bot protection or CDN. - Yes: Then it can check a signed request against the agent’s published key, which the draft says is meant to let servers “verify their identity with confidence”. The draft is at revision 00, so it can still change. Source: [IETF draft-ietf-webbotauth-httpsig-protocol-00](https://datatracker.ietf.org/doc/draft-ietf-webbotauth-httpsig-protocol/). Read on: [Bots, crawlers and agents: who is knocking](https://storewalk.ai/blog/bots-crawlers-agents). - No: Then it has to identify agents by other means, such as “IP allowlisting, User-Agent strings, or shared API keys”, which the draft says “have significant limitations”. RFC 9309 adds that robots.txt rules “are not a form of access authorization”. Sources: [IETF draft-ietf-webbotauth-httpsig-protocol-00](https://datatracker.ietf.org/doc/draft-ietf-webbotauth-httpsig-protocol/) and [RFC 9309: Robots Exclusion Protocol](https://www.rfc-editor.org/rfc/rfc9309). Read on: [Bots, crawlers and agents: who is knocking](https://storewalk.ai/blog/bots-crawlers-agents). - Not sure: Then find out first; choose Yes or No to see what each case means for an agent. Read on: [Bots, crawlers and agents: who is knocking](https://storewalk.ai/blog/bots-crawlers-agents). 3. Finding what the store offers. Does https\://your-domain/.well-known/ucp return a JSON document with a ucp member? Open that address in a browser, with your store’s domain in it. - Yes: Then something publishes a UCP profile for your store. UCP says “Businesses publish their profile at /.well-known/ucp”, as a JSON object “with a required ucp member”. Source: [UCP specification overview](https://ucp.dev/latest/specification/overview/). Read on: [How agents discover what a store offers](https://storewalk.ai/blog/how-agents-discover-a-store) and [UCP explained](https://storewalk.ai/blog/ucp-explained). - No: Then an agent finds no UCP profile at the address UCP gives. ACP has no released file to add instead: its discovery file is a proposal, marked “Version: unreleased”. Sources: [UCP specification overview](https://ucp.dev/latest/specification/overview/) and [ACP RFC: Discovery (Proposal, unreleased)](https://github.com/agentic-commerce-protocol/agentic-commerce-protocol/blob/main/rfcs/rfc.discovery.md). Read on: [How agents discover what a store offers](https://storewalk.ai/blog/how-agents-discover-a-store) and [UCP explained](https://storewalk.ai/blog/ucp-explained). - Not sure: Then find out first; choose Yes or No to see what each case means for an agent. Read on: [How agents discover what a store offers](https://storewalk.ai/blog/how-agents-discover-a-store) and [UCP explained](https://storewalk.ai/blog/ucp-explained). 4. Getting through checkout. Does your platform implement the UCP or the ACP checkout for your store? Ask your platform, or whoever maintains your checkout. - Yes: Then an agent can create, update and complete a checkout through requests instead of pages. When UCP needs the buyer, the store “MUST provide continue_url” with the status requires_escalation. Sources: [UCP checkout capability](https://ucp.dev/latest/specification/shopping/checkout/) and [ACP Agentic Checkout API, 2026-04-17](https://github.com/agentic-commerce-protocol/agentic-commerce-protocol/blob/main/spec/2026-04-17/openapi/openapi.agentic_checkout.yaml). Read on: [Why agents stall at checkout](https://storewalk.ai/blog/why-agents-stall-at-checkout), [UCP explained](https://storewalk.ai/blog/ucp-explained) and [ACP explained](https://storewalk.ai/blog/acp-explained). - No: In our reading, an agent then has only your checkout pages, the ones built for a person at a browser. UCP’s Create, Get, Update, Complete and Cancel, and ACP’s checkout session endpoints, are not there for it to call. Sources: [UCP checkout capability](https://ucp.dev/latest/specification/shopping/checkout/) and [ACP Agentic Checkout API, 2026-04-17](https://github.com/agentic-commerce-protocol/agentic-commerce-protocol/blob/main/spec/2026-04-17/openapi/openapi.agentic_checkout.yaml). Read on: [Why agents stall at checkout](https://storewalk.ai/blog/why-agents-stall-at-checkout), [UCP explained](https://storewalk.ai/blog/ucp-explained) and [ACP explained](https://storewalk.ai/blog/acp-explained). - Not sure: Then find out first; choose Yes or No to see what each case means for an agent. Read on: [Why agents stall at checkout](https://storewalk.ai/blog/why-agents-stall-at-checkout), [UCP explained](https://storewalk.ai/blog/ucp-explained) and [ACP explained](https://storewalk.ai/blog/acp-explained). 5. Paying on someone’s behalf. Does your payment provider accept ACP delegated tokens or Stripe Shared Payment Tokens? Ask your payment provider. - Yes: Then an agent can pay with a token instead of the card. ACP’s token is limited by an allowance: a maximum amount, a currency, an expiry, one checkout session and one merchant. Stripe offers its tokens as a preview. Sources: [ACP Delegate Payment API, 2026-04-17](https://github.com/agentic-commerce-protocol/agentic-commerce-protocol/blob/main/spec/2026-04-17/openapi/openapi.delegate_payment.yaml) and [Stripe docs: Shared payment tokens](https://docs.stripe.com/agentic-commerce/concepts/shared-payment-tokens). Read on: [How an agent pays without seeing your card](https://storewalk.ai/blog/how-agents-pay) and [ACP explained](https://storewalk.ai/blog/acp-explained). - No: Then an agent has no ACP or Stripe token your provider takes. ACP’s token is meant for “the merchant’s PSP”. AP2 works differently: its Payment Mandate goes to the credential provider, the networks and the merchant’s payment processor. Sources: [ACP Delegate Payment API, 2026-04-17](https://github.com/agentic-commerce-protocol/agentic-commerce-protocol/blob/main/spec/2026-04-17/openapi/openapi.delegate_payment.yaml) and [AP2 specification](https://ap2-protocol.org/ap2/specification/). Read on: [How an agent pays without seeing your card](https://storewalk.ai/blog/how-agents-pay) and [ACP explained](https://storewalk.ai/blog/acp-explained). - Not sure: Then find out first; choose Yes or No to see what each case means for an agent. Read on: [How an agent pays without seeing your card](https://storewalk.ai/blog/how-agents-pay) and [ACP explained](https://storewalk.ai/blog/acp-explained). Several of the documents behind these walls are early or not yet settled: - Web Bot Auth is a working-group Internet-Draft at revision 00. - llms.txt is a proposal, and Markdown for Agents is one company's feature. - ACP calls itself beta, and its discovery file is a proposal. - Stripe's Shared Payment Tokens are a preview. - AP2 is at version 0.2, and its mandate types changed from version 0.1. - UCP refers to a "UCP governing body" without naming it. ## Sources - [The /llms.txt file, v2](https://llmstxt.org/) - [Cloudflare docs: Markdown for Agents](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/) - [IETF draft-ietf-webbotauth-httpsig-protocol-00: HTTP Message Signatures for automated traffic](https://datatracker.ietf.org/doc/draft-ietf-webbotauth-httpsig-protocol/) - [RFC 9421: HTTP Message Signatures](https://www.rfc-editor.org/rfc/rfc9421) - [RFC 9309: Robots Exclusion Protocol](https://www.rfc-editor.org/rfc/rfc9309) - [OpenAI: Overview of OpenAI crawlers](https://developers.openai.com/api/docs/bots) - [Google: JavaScript SEO basics](https://developers.google.com/search/docs/crawling-indexing/javascript/javascript-seo-basics) - [UCP specification overview](https://ucp.dev/latest/specification/overview/) - [UCP checkout capability](https://ucp.dev/latest/specification/shopping/checkout/) - [ACP RFC: Discovery](https://github.com/agentic-commerce-protocol/agentic-commerce-protocol/blob/main/rfcs/rfc.discovery.md) - [Agentic Checkout API, 2026-04-17](https://github.com/agentic-commerce-protocol/agentic-commerce-protocol/blob/main/spec/2026-04-17/openapi/openapi.agentic_checkout.yaml) - [ACP RFC: Agentic Checkout](https://github.com/agentic-commerce-protocol/agentic-commerce-protocol/blob/main/rfcs/rfc.agentic_checkout.md) - [RFC 8414: OAuth 2.0 Authorization Server Metadata](https://www.rfc-editor.org/rfc/rfc8414) - [RFC 9728: OAuth 2.0 Protected Resource Metadata](https://www.rfc-editor.org/rfc/rfc9728) - [ACP Delegate Payment API, 2026-04-17](https://github.com/agentic-commerce-protocol/agentic-commerce-protocol/blob/main/spec/2026-04-17/openapi/openapi.delegate_payment.yaml) - [ACP RFC: Payment Handlers](https://github.com/agentic-commerce-protocol/agentic-commerce-protocol/blob/main/rfcs/rfc.payment_handlers.md) - [Stripe docs: Shared payment tokens](https://docs.stripe.com/agentic-commerce/concepts/shared-payment-tokens) - [AP2 specification](https://ap2-protocol.org/ap2/specification/) - [Google: Agent Payments Protocol and the FIDO Alliance](https://blog.google/products-and-platforms/platforms/google-pay/agent-payments-protocol-fido-alliance/) - [agenticcommerce.dev](https://www.agenticcommerce.dev/) - [ACP repository and README](https://github.com/agentic-commerce-protocol/agentic-commerce-protocol) Last checked 22 Oct 2026