Mistral's AI crawlers
Mistral documents each of these and says what it is for, so a site can say yes to one and no to another. Numbers are from the October 2026 study of the top 1,000 websites (2026-10-12).
The split
| Token | Job | Score | Blocked by | Allowed but refused at the server |
|---|---|---|---|---|
MistralAI-User | Fetches when a user asks | Counted | 11.8% | not tested |
MistralAI-Index | AI search index | Counted | — | not tested |
MistralAI-Training | Training only | Not scored | — | not tested |
“Blocked by” is the share of the 569 top-1,000 sites with a robots.txt. The last column was measured for four crawlers only.
MistralAI-User · Fetches when a user asks
Visits a page when a user's question in Mistral's assistant needs it, and links to the source; Mistral says it is not used for crawling the web automatically or for training.
Documented by the vendor. Its page has the robots.txt lines and what blocking costs.
MistralAI-Index · AI search index
Crawls for indexing only, feeding the search that answers questions in Mistral's assistant. Mistral says what it crawls is excluded from generative AI training.
Documented by the vendor. Its page has the robots.txt lines and what blocking costs.
MistralAI-Training · Training only
Crawls web content to build the datasets that train Mistral's generative models; not used for search or live answers. Mistral says a robots.txt disallow keeps a site out.
Documented by the vendor. Its page has the robots.txt lines and what blocking costs.
Opt out of Mistral's training, stay in its answers
The lines below block only the crawler that gathers training data. MistralAI-User and MistralAI-Index stay open, so the site stays in Mistral's answers. The readiness score does not change.
User-agent: MistralAI-Training
Disallow: /Which of Mistral's crawlers can read your site?
Run the free check, or write the whole policy with the robots.txt generator.