Amazon's AI crawlers
Amazon documents each of these and says what it is for, so a site can say yes to one and no to another. Numbers are from the October 2026 study of the top 1,000 websites (2026-10-12).
The split
| Token | Job | Score | Blocked by | Allowed but refused at the server |
|---|---|---|---|---|
Amazonbot | AI answers | Counted | 13.7% | not tested |
Amzn-SearchBot | AI search index | Counted | — | not tested |
Amzn-User | Fetches when a user asks | Counted | — | not tested |
“Blocked by” is the share of the 569 top-1,000 sites with a robots.txt. The last column was measured for four crawlers only.
Amazonbot · AI answers
Improves Amazon's products and services and may be used to train Amazon AI models. Amazon also treats a noarchive meta tag as a request not to use the page for model training.
Documented by the vendor. Its page has the robots.txt lines and what blocking costs.
Amzn-SearchBot · AI search index
Improves search in Amazon's products, so allowed pages can appear in places like Alexa; Amazon says it does not crawl for generative AI training. When robots.txt does not name it but allows other search crawlers, it follows the rules given to those.
Documented by the vendor. Its page has the robots.txt lines and what blocking costs.
Amzn-User · Fetches when a user asks
Fetches live information when a user asks, such as an Alexa question that needs current facts; not used for training. Because a user starts it, Amazon says it may not follow every robots.txt directive.
Documented by the vendor. Its page has the robots.txt lines and what blocking costs.
Which of Amazon's crawlers can read your site?
Run the free check, or write the whole policy with the robots.txt generator.