Cohere's AI crawlers
Cohere publishes no crawler documentation; these names come from community lists, so treat them as what people write in robots.txt rather than as the vendor's promise. Numbers are from the October 2026 study of the top 1,000 websites (2026-10-12).
The split
| Token | Job | Score | Blocked by | Allowed but refused at the server |
|---|---|---|---|---|
cohere-ai | AI answers | Counted | 15.8% | not tested |
cohere-training-data-crawler | Training only | Not scored | 12.5% | not tested |
“Blocked by” is the share of the 569 top-1,000 sites with a robots.txt. The last column was measured for four crawlers only.
cohere-ai · AI answers
Cohere's crawler page says it does not currently use bots to crawl or scrape the web for training, and names no active user agent. This token comes from community lists.
Community-listed; no vendor documentation. Its page has the robots.txt lines and what blocking costs.
cohere-training-data-crawler · Training only
Listed by community directories as a Cohere training crawler. Cohere's own page says it is not currently crawling for training and that any future crawler will respect robots.txt.
Community-listed; no vendor documentation. Its page has the robots.txt lines and what blocking costs.
Opt out of Cohere's training, stay in its answers
The lines below block only the crawler that gather training data. cohere-ai stays open, so the site is still found and cited. The readiness score does not change.
User-agent: cohere-training-data-crawler
Disallow: /Which of Cohere's crawlers can read your site?
Run the free check, or write the whole policy with the robots.txt generator.