Skip to content
Sign in

SAFE-101Beside the path

Spoofed agent UA accepted

Checks whether a request that only claims to be a well-known AI agent, through a forged user-agent, gets the same content from your site as anyone else.

What we check

This active safety probe runs only when you have verified you own the domain and asked for the full safety test, never on a public scan. We send a GET to your product page, cart and checkout carrying the user-agent string of a well-known agent (for example ChatGPT-User, ClaudeBot or PerplexityBot) and no signature, from ordinary non-vendor IP addresses, beside a baseline request that uses our own crawler user-agent. A user-agent string is trivial to forge, so this measures whether that claim alone is enough to be treated as a genuine agent.

Why it matters

Agent user-agents are self-reported and unauthenticated: any scraper can set one. If a forged agent UA gets the same content as a shopper, then "allowing agents" in practice means "allowing anyone who says they are an agent". Only a signed request (Web Bot Auth) reliably tells a genuine agent from an impostor.

How we check

For each of the product page, cart and checkout we compare the spoofed-agent response with the baseline. A target where our own baseline request failed (our budget or timeout, or the shop erroring) counts neither way.

PASS: on at least one page the spoofed agent was challenged, blocked, or served clearly reduced content (under 60% of the baseline body). FAIL (HIGH): every comparable page returned identical full 200 responses to the forged agent. SKIPPED: no product, cart or checkout URL was available to probe, or the shop answered none of the requests. The probe is read-only: it sends GET requests only and never adds to a cart or submits a form.

How to fix it

  1. Do not treat an agent user-agent as identity: it is self-reported and forgeable.
  2. For anything you grant agents, require a verified signal, such as a request signed with Web Bot Auth or one from an operator's published IP ranges, instead of a UA-string match.
  3. In bot management, challenge unsigned traffic that claims a known agent UA on cart, checkout and search, while allowing verified agents through.

What good looks like

# Baseline (our crawler UA)
GET /products/anvil            -> HTTP/2 200  (42 KB)

# Forged agent UA, unsigned
GET /products/anvil
user-agent: ClaudeBot/1.0
                               -> HTTP/2 403  (managed challenge)

A genuine agent would instead present a Web Bot Auth signature and be allowed in.

Sources

Read more

  • Bots, crawlers and agents: who is knocking

    A User-Agent string can be spoofed, and robots.txt rules are a request to crawlers rather than access authorization. Web Bot Auth, an IETF draft, would let an agent sign its requests so a server can verify who sent them.

Could an agent buy from your store?

The free scan runs all 51 checks on your store in about five minutes.

Scan my store
The storewalk.ai app on a phone, scanning a clothing store: it checks a product, views its details and compares options.