Skip to content

SAFE-101Beside the path

Spoofed agent UA accepted

Checks whether your site serves the same content to a request that merely claims to be a well-known AI agent as it does to anyone else — a forged agent user-agent that walks straight in.

What we check

This is one of the active safety probes: it runs only when you have verified you own the domain and asked for the full safety test, never on a public scan. We send a GET to your product page, cart and checkout carrying the user-agent string of a well-known agent (for example ChatGPT-User, ClaudeBot or PerplexityBot) and no signature, from ordinary non-vendor IP addresses, beside a baseline request that uses our own crawler user-agent. A user-agent string is trivial to forge, so this measures whether that claim alone is enough to be treated as a genuine agent.

Why it matters

Agent user-agents are self-reported and unauthenticated: any scraper can set one. If a forged agent UA is served the same content as a shopper, then "allowing agents" has quietly become "allowing anyone who says they are an agent". The only reliable way to tell a genuine agent from an impostor is a signed request (Web Bot Auth), not the UA string.

How we check

For each of the product page, cart and checkout we compare the spoofed-agent response with the baseline. A target where our own baseline request failed (our budget or timeout, or the shop erroring) is not counted either way.

PASS: on at least one page the spoofed agent was challenged, blocked, or served clearly reduced content (under 60% of the baseline body). FAIL (HIGH): every comparable page returned identical full 200 responses to the forged agent. SKIPPED: no product, cart or checkout URL was available to probe, or the shop answered none of the requests. Read-only — GET only, it never adds to a cart or submits a form.

How to fix it

  1. Do not treat an agent user-agent as identity: it is self-reported and forgeable.
  2. Require verified signals for anything you grant agents — Web Bot Auth signed requests, or an operator’s published IP ranges — rather than a UA-string match.
  3. In bot management, challenge unsigned traffic that claims a known agent UA on cart, checkout and search, while allowing verified agents through.

What good looks like

# Baseline (our crawler UA)
GET /products/anvil            -> HTTP/2 200  (42 KB)

# Forged agent UA, unsigned
GET /products/anvil
user-agent: ClaudeBot/1.0
                               -> HTTP/2 403  (managed challenge)

A genuine agent would instead present a Web Bot Auth signature and be allowed in.

Sources