# Markdown for agents: serve your pages to AI How to give AI agents a Markdown copy of your pages, with Cloudflare Markdown for Agents or on your own stack, and how to check the answer your server gives. By Valerii Checha, 4 Nov 2026 What is Markdown for agents? It is a plain-text copy of a page, served to an AI agent that asks for it with `Accept: text/markdown`, while people keep getting HTML. On Cloudflare's Pro, Business or Enterprise plans you can switch it on and Cloudflare converts your pages at its edge, up to 6 MiB of HTML each. Anywhere else you serve the Markdown yourself, from the same address or from a `.md` URL, labelled `text/markdown; charset=utf-8` with `Vary: Accept`. This guide covers why an agent would want Markdown, how to turn it on at Cloudflare, how to build it yourself, and how to check the answer your server gives, with an Accept header simulator, a converter and a response checker that run in this page. Storewalk is ours: we build the scan, and its [Markdown check](https://storewalk.ai/checks/DISC-009) is ours. Cloudflare, Shopify, Vercel and the RFCs are quoted from their own pages, read on 4 November 2026. For how agents read a page in the first place, read [how agents read a page](https://storewalk.ai/blog/how-agents-read-a-page), and for the five walls between an agent and a sale, [the hub article](https://storewalk.ai/blog/why-agents-cant-buy). ## Why would an AI agent want Markdown? Because a product page carries a lot besides the product. Navigation, scripts, styles and tracking sit around the description, and an agent has to read past them to find it. Cloudflare's converter drops "Non-content elements (such as headers, footers, navigation, scripts, and styles)" and keeps the page's JSON-LD at the end as "a fenced json code block". Shopify's Help Center, which serves its own pages this way, says the result "drops site chrome and navigation, leaving just the page body, which is the preferred representation for LLM ingestion". In our words, the agent gets the shop window without the scaffolding. Markdown is not a replacement for structured data. Your prices and stock should still be in the Product markup that the [structured data guide](https://storewalk.ai/guides/product-structured-data-for-ai-agents) covers, and Cloudflare carries that markup across. Markdown helps an agent read the prose around it: descriptions, sizes, delivery and returns. ## How does an agent ask for Markdown? With the `Accept` header, the ordinary HTTP way of asking for one format over another. Cloudflare says a client "needs to add the Accept negotiation header with text/markdown as one of the options". The server answers with `Content-Type: text/markdown`, and because the same address now has two answers, the response says so in `Vary`. Cloudflare's page puts it this way: "Vary includes Accept (any Vary dimensions your origin already declared are preserved) so that caches store separate variants for Markdown and HTML." The `text/markdown` type comes from RFC 7763, an Informational RFC of March 2016, which says "charset is REQUIRED". Servers differ in how strictly they read the header. Try a few: Ask for Markdown: what four kinds of server send back. On the page, you pick a server and an Accept header and see what comes back. Every pair: Cloudflare, Markdown for Agents on. Cloudflare converts when the Accept header has text/markdown “as one of the options”, and sets “Vary includes Accept”. - Accept: text/markdown: Markdown, Content-Type text/markdown; charset=utf-8. text/markdown is one of the options, so Cloudflare converts the HTML. - Accept: text/markdown, text/html: Markdown, Content-Type text/markdown; charset=utf-8. text/markdown is one of the options, so Cloudflare converts the HTML. - Accept: text/markdown;q=0.5, text/html: Markdown, Content-Type text/markdown; charset=utf-8. text/markdown is one of the options. Cloudflare’s page does not say how it weighs q values; we show the conversion. - Accept: text/html: HTML, Content-Type text/html; charset=utf-8. No text/markdown in the header, so the origin’s HTML passes through. - Accept: \*/\*: HTML, Content-Type text/html; charset=utf-8. No text/markdown in the header, so the origin’s HTML passes through. Shopify’s Help Center rules. Its own rules: the type must be named, “Markdown wins an equal-weight tie”, and only charset=utf-8 is honoured. - Accept: text/markdown: Markdown, Content-Type text/markdown; charset=utf-8. text/markdown is the preferred type. - Accept: text/markdown, text/html: Markdown, Content-Type text/markdown; charset=utf-8. An equal-weight tie, which Markdown wins. - Accept: text/markdown;q=0.5, text/html: HTML, Content-Type text/html; charset=utf-8. text/html has the higher q value, so HTML wins. - Accept: text/html: HTML, Content-Type text/html; charset=utf-8. No text/markdown in the header: HTML. - Accept: \*/\*: HTML, Content-Type text/html; charset=utf-8. The media type must be named explicitly; \*/\* and text/\* return HTML. A server with no negotiation. The page comes back as HTML whatever the request asks for. - Accept: text/markdown: HTML, Content-Type text/html; charset=utf-8. No negotiation: HTML for every request. - Accept: text/markdown, text/html: HTML, Content-Type text/html; charset=utf-8. No negotiation: HTML for every request. - Accept: text/markdown;q=0.5, text/html: HTML, Content-Type text/html; charset=utf-8. No negotiation: HTML for every request. - Accept: text/html: HTML, Content-Type text/html; charset=utf-8. No negotiation: HTML for every request. - Accept: \*/\*: HTML, Content-Type text/html; charset=utf-8. No negotiation: HTML for every request. A Shopify storefront, as we saw it. Our observation of three Shopify stores on 24 September 2026, not re-run since: a Markdown label on the HTML page. - Accept: text/markdown: HTML, Content-Type text/markdown. A Markdown label on the full HTML page, as we observed. - Accept: text/markdown, text/html: HTML, Content-Type text/markdown. A Markdown label on the full HTML page, as we observed. - Accept: text/markdown;q=0.5, text/html: HTML, Content-Type text/markdown. A Markdown label on the full HTML page, as we observed. - Accept: text/html: HTML, Content-Type text/html; charset=utf-8. No text/markdown asked for: the HTML page. - Accept: \*/\*: HTML, Content-Type text/html; charset=utf-8. No text/markdown asked for: the HTML page. Sources: [Cloudflare: Markdown for Agents](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/), [Shopify Help Center: agents.md](https://help.shopify.com/agents.md) and [Our check: pages served as Markdown on request (DISC-009)](https://storewalk.ai/checks/DISC-009). Shopify's Help Center is the strict one, and its rules are a good model: "The media type must be named explicitly", `*/*` returns HTML, "Markdown wins an equal-weight tie", and "The only media-type parameter honoured is charset=utf-8". Those are the rules of Shopify's documentation site. A Shopify storefront is another matter: when we tested three Shopify stores on 24 September 2026, each answered `Accept: text/markdown` with a `text/markdown` label on the full HTML page. We have not re-run that test since. ## How do you turn on Markdown for Agents at Cloudflare? In the dashboard. Cloudflare's steps are to select your zone ("you need a Pro or Business plan"), visit the AI Crawl Control section and enable Markdown for Agents. You can also switch it on for one subdomain or path with a configuration rule, or through the API with a `PATCH` to `/client/v4/zones/{zone_tag}/settings/content_converter` and `{"value": "on"}`. Cloudflare says it "is available to Pro, Business and Enterprise plans, and SSL for SaaS customers at no cost", so the Free plan is not on the list. Cloudflare's page also sets limits and a default: - The size limit: "Conversion is limited to 6 MiB (6,291,456 bytes) of decompressed HTML", and "The limit applies after decompression, not to the compressed response size." - Only HTML: "We only convert from HTML, other types of documents may be included in the future." - No rendering: it converts the HTML your origin sends. For pages built in the browser, Cloudflare points to its Browser Run `/markdown` endpoint, "if you need to render a dynamic page or application in a real browser before converting it". - A default policy: if your origin sends no content signal, Cloudflare "adds a default Content-Signal: ai-train=yes, search=yes, ai-input=yes". If you want a different answer for training, send your own; the [robots.txt guide](https://storewalk.ai/guides/robots-txt-for-ai-agents) covers content signals. ## How do you serve Markdown without Cloudflare? Build it into your site. Vercel describes the pattern for Next.js: "The implementation has two parts, a rewrite rule in next.config.ts that detects the header and a route handler that returns markdown." Any framework can do the same: look at `Accept`, and when it names `text/markdown`, return a Markdown version of the page with the right headers. Shopify's Help Center adds a second route and prefers it: "Append .md to the URL." Its reasons are practical. "The .md URL is distinct from the HTML URL, so responses are cache-safe", and "A .md request for a page that doesn't exist returns 404", where a negotiated request may redirect instead. Our own site does both: this guide answers `Accept: text/markdown` at its own address and also lives at `/guides/markdown-for-ai-agents.md`. Link the `.md` versions from your [llms.txt file](https://storewalk.ai/guides/llms-txt-for-online-stores) so agents can find them. Paste a page's HTML to see roughly what a converter keeps: Your page as Markdown, converted in this page. On the page, you paste a page's HTML and see it as Markdown, converted by our own small converter in the layout Cloudflare documents: front matter from the title and description, header, footer, navigation and scripts left out, JSON-LD kept at the end. An invented product page: ```html Linen duvet cover | Northwind Linen

Linen duvet cover

Stonewashed European linen, £120.00. In stock.

Sizes

Delivery and returns

Free UK delivery in 3 to 5 working days. Returns within 30 days, see our refund policy.

Oatmeal linen duvet cover on a bed
``` As Markdown: ````markdown --- title: Linen duvet cover | Northwind Linen description: Stonewashed linen duvet cover in four sizes. --- # Linen duvet cover Stonewashed European linen, **£120.00**. In stock. ## Sizes - Single - Double - King - Super king ## Delivery and returns Free UK delivery in 3 to 5 working days. Returns within 30 days, see [our refund policy](/policies/refund-policy). ![Oatmeal linen duvet cover on a bed](/img/duvet.jpg) ```json {"@context":"https://schema.org","@type":"Product","name":"Linen duvet cover","offers":{"@type":"Offer","price":"120.00","priceCurrency":"GBP","availability":"https://schema.org/InStock"}} ``` ```` HTML 1064 bytes, Markdown 629 bytes, within Cloudflare's 6 MiB conversion limit. Source: [Cloudflare: Markdown for Agents](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/). If the Markdown comes out empty, your page is probably built by JavaScript in the browser, and so is what an agent that does not run scripts sees: nothing much. ## What should a store's Markdown page say? What the HTML page says, minus the furniture. This part is our advice rather than anyone's rule. For a product page that means: - the product's name as the first heading, and its price written with the currency, as in "£120.00", not a bare number; - the options a shopper chooses between (sizes, colours) and whether each is in stock; - a line on delivery and returns, with links to the policy pages, ideally to their own `.md` versions; - the page's canonical address, so an agent that quotes you links to the real page. Keep the Markdown to what the HTML shows. A Markdown copy that promises a faster delivery or a lower price than the page invites an agent to repeat a claim your checkout will not honour, and the [shipping and returns guide](https://storewalk.ai/guides/shipping-returns-policies-for-agents) covers keeping those terms the same in every place you state them. Generating both from the same product data keeps them in step. ## How do you check that your Markdown works? Ask for it and read the answer. Run `curl -si -H 'Accept: text/markdown' https://your-store/` and paste the output here: Check a Markdown response, and what our DISC-009 says. On the page, you paste the output of curl -si -H 'Accept: text/markdown' for your home page and see what is right, and what our DISC-009 says. The examples: Markdown, done right: ```http HTTP/2 200 content-type: text/markdown; charset=utf-8 vary: Accept cache-control: public, max-age=3600 --- title: Northwind Linen --- # Bedding made of linen ``` - Yes: Answers 200. - Yes: Content-Type is text/markdown. - Yes: It names a charset, which RFC 7763 makes required. - Yes: Vary includes Accept, so caches keep the HTML and Markdown apart. - Yes: The body is Markdown, not the HTML page. - DISC-009: PASS, answers with a Markdown body. A label on the HTML page: ```http HTTP/2 200 content-type: text/markdown vary: Accept Northwind Linen ``` - Yes: Answers 200. - Yes: Content-Type is text/markdown. - No: It names a charset, which RFC 7763 makes required. - Yes: Vary includes Accept, so caches keep the HTML and Markdown apart. - No: The body is Markdown, not the HTML page. - DISC-009: WARN 45, labelled text/markdown, but the body is the HTML page. No negotiation: ```http HTTP/2 200 content-type: text/html; charset=utf-8 ``` - Yes: Answers 200. - No: Content-Type is text/markdown. - No: It names a charset, which RFC 7763 makes required. - No: Vary includes Accept, so caches keep the HTML and Markdown apart. - No: The body is Markdown, not the HTML page. - DISC-009: FAIL 0, ignores Accept: text/markdown (HTTP 200, text/html). No Vary, no charset: ```http HTTP/1.1 200 OK content-type: text/markdown # Bedding made of linen ``` - Yes: Answers 200. - Yes: Content-Type is text/markdown. - No: It names a charset, which RFC 7763 makes required. - No: Vary includes Accept, so caches keep the HTML and Markdown apart. - Yes: The body is Markdown, not the HTML page. - DISC-009: PASS, answers with a Markdown body. Sources: [RFC 7763: The text/markdown Media Type](https://www.rfc-editor.org/rfc/rfc7763), [Cloudflare: Markdown for Agents](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/) and [Our check: pages served as Markdown on request (DISC-009)](https://storewalk.ai/checks/DISC-009). Our [DISC-009](https://storewalk.ai/checks/DISC-009) sends one such request to your home page. It passes a `text/markdown` answer whose body is Markdown, warns when the label sits on the HTML page or on an empty body, fails when the answer is HTML, and skips when a bot wall or a server error answers instead. It reads the label and the body, and leaves the charset and `Vary` to you, which is why the fourth example passes our check and still misses two things RFC 7763 and caching ask for. ## Do AI agents actually ask for Markdown? Some say they do, and nobody publishes a count we could check. Vercel writes that "Many agents already send Accept: text/markdown when fetching pages". The crawler pages of OpenAI and Anthropic that we read describe their bots and robots.txt, and say nothing about the `Accept` header. So treat Markdown as a cheap courtesy: it costs a setting on Cloudflare or a route handler on your own stack, and an agent that asks gets a cleaner page. ## Frequently asked questions ### What is Cloudflare Markdown for Agents? A Cloudflare setting that converts your HTML to Markdown at the edge when a request asks for `text/markdown`. It "is available to Pro, Business and Enterprise plans, and SSL for SaaS customers at no cost". ### What is the size limit for Markdown for Agents? "Conversion is limited to 6 MiB (6,291,456 bytes) of decompressed HTML", measured after decompression. ### Which headers should a Markdown response have? `Content-Type: text/markdown; charset=utf-8`, since RFC 7763 says "charset is REQUIRED", and `Vary: Accept`, so caches keep the HTML and Markdown versions apart. ### Does Shopify serve Markdown for product pages? Shopify's own Help Center does, at `.md` URLs and through `Accept`. When we tested three storefronts on 24 September 2026, they put a `text/markdown` label on the HTML page, which our DISC-009 reports as a warning. ### Do I still need structured data if I serve Markdown? Yes. Markdown carries the prose, and Product markup carries the price, currency and stock in fields an agent can trust. Cloudflare keeps your JSON-LD at the end of its Markdown, so the two travel together. Our [agent readiness scanner](https://storewalk.ai/) checks both. The sources below were last checked on the date shown. ## Sources - [Cloudflare: Markdown for Agents](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/) - [RFC 7763: The text/markdown Media Type](https://www.rfc-editor.org/rfc/rfc7763) - [Shopify Help Center: agents.md](https://help.shopify.com/agents.md) - [Vercel: Making agent-friendly pages with content negotiation](https://vercel.com/blog/making-agent-friendly-pages-with-content-negotiation) - [OpenAI: Overview of OpenAI crawlers](https://developers.openai.com/api/docs/bots) - [Anthropic: Does Anthropic crawl data from the web?](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) Last checked 4 Nov 2026