Skip to content

SAFE-104Beside the path

Catalog enumeration

Checks whether sequential product IDs can be walked unthrottled, letting anyone scrape your whole catalogue just by counting upward.

What we check

An active safety probe in the owner-requested safety test. Starting from a real product URL, we take the last integer in its path and request the next 50 sequential IDs — following your own numbering rather than inventing a scheme — with our crawler user-agent. We measure how many resolve before anything throttles us.

Why it matters

If /products/1001 leads to 1002, 1003 and on with no limit, your entire catalogue — including unlisted or not-yet-launched items — is trivially scrapable by a script that just counts. Throttling, or non-guessable identifiers, stops that.

How we check

We stop at the first challenge and never count our own budget or timeout as a throttle.

WARN (the only scored negative outcome the spec defines for this probe): 40 or more of the 50 sequential IDs resolved with no throttle — the catalogue is trivially enumerable. PASS: a throttle or challenge appeared mid-walk (you do limit enumeration), or too few IDs resolved (your live catalogue does not follow the guessed sequence). SKIPPED: no product URL to start from, the URL carries no numeric id to walk, or nothing was answered. This probe never FAILs — the spec defines only the WARN condition. Read-only GETs.

How to fix it

  1. Rate-limit sequential access to product URLs per client, as you would for search.
  2. Prefer non-sequential, non-guessable product identifiers (handles or opaque slugs) over plain incrementing integers.
  3. Return a challenge or a 404 for out-of-range or not-yet-published IDs rather than serving them.

What good looks like

Starting from /products/1001, a shop that guards enumeration:

GET /products/1002 -> 200
GET /products/1003 -> 200
...
GET /products/1020 -> HTTP 429 / managed challenge

The walk is throttled long before the whole catalogue can be counted out.

Sources