AI Readiness Report
reuters.com
Restricted by Design
◆70/100
This site restricts most AI crawlers by choice — 33 of 35 AI search and answer crawlers are blocked in robots.txt.
This site appears to intentionally restrict AI crawler access — a valid strategy many publishers and brands choose (for example, keeping content out of AI training). If that’s deliberate, the site is working as intended: the score below measures how open a site is to AI, so a low number here is the expected result of that choice — not a failure to fix.
2
Allowed
33
Blocked
2
Fixes
Blocked crawlers
- Claude-UserAnthropic
- Claude-SearchBotAnthropic
- Google-ExtendedGoogle AI
- Gemini-Deep-ResearchGoogle AI
- Google-NotebookLMGoogle AI
- PerplexityBotPerplexity
- Perplexity-UserPerplexity
- GrokBotxAI / Grok
Recommended fixes
- AI crawlers appear to be intentionally restricted (33 of 35 AI search and answer crawlers blocked in robots.txt)Impact · low
- Your homepage returned HTTP 401 to our crawler — homepage checks were skippedImpact · info
Optional extra
Your site has no llms.txt file
A Markdown summary of reuters.com for AI systems. Not scored, since few AI crawlers ask for it yet, but quick to add.
Quick fix available
33 AI crawlers blocked by robots.txt
Generate an AI-crawler-aware robots.txt for reuters.com — pick a preset, download the file.