Meta AI's AI crawlers
Meta AI documents 2 of these 3; the others are names from community lists. Numbers are from the October 2026 study of the top 1,000 websites (2026-10-12).
The split
| Token | Job | Score | Blocked by | Allowed but refused at the server |
|---|---|---|---|---|
meta-externalagent | AI answers | Counted | 16.9% | not tested |
meta-externalfetcher | Fetches when a user asks | Counted | 11.2% | not tested |
FacebookBot | Training only | Not scored | 13.4% | not tested |
“Blocked by” is the share of the 569 top-1,000 sites with a robots.txt. The last column was measured for four crawlers only.
meta-externalagent · AI answers
Crawls for training Meta's foundation AI models or improving its products by indexing content directly. Meta's documented way to block it is a robots.txt disallow.
Documented by the vendor. Its page has the robots.txt lines and what blocking costs.
meta-externalfetcher · Fetches when a user asks
Fetches individual links at a user's request for Meta's agentic AI features. Meta says it may bypass robots.txt.
Documented by the vendor. Its page has the robots.txt lines and what blocking costs.
FacebookBot · Training only
Meta's current crawler page does not list it. Community lists say it gathered data for Meta's speech-recognition work. Kept so a robots.txt that names it is read correctly; it is not scored.
Community-listed; no vendor documentation. Its page has the robots.txt lines and what blocking costs.
Opt out of Meta AI's training, stay in its answers
The lines below block only the crawler that gather training data. meta-externalagent, meta-externalfetcher stay open, so the site is still found and cited. The readiness score does not change.
User-agent: FacebookBot
Disallow: /Which of Meta AI's crawlers can read your site?
Run the free check, or write the whole policy with the robots.txt generator.