[AI crawlers](https://www.crawlreadiness.com/crawlers) · Cohere · Training only · Not scored

# cohere-training-data-crawler

cohere-training-data-crawler is Cohere's training only crawler token. Blocking it costs nothing in the AI readiness score. 12.5% of the top 1,000 websites with a robots.txt block it.

Listed by community directories as a Cohere training crawler. Cohere's own page says it is not currently crawling for training and that any future crawler will respect robots.txt.

## Where this comes from

- **Community-listed; no vendor documentation.** Source: [docs.cohere.com](https://docs.cohere.com/docs/cohere-web-crawlers)
- **Robots.txt behaviour undocumented.**
- **Listed, not scored.** It cannot keep a site out of today's answers, so the check shows a block without deducting for it.

## In the top 1,000 websites

- **12.5% block it in robots.txt** (71 of the 569 sites with a robots.txt), as of 2026-10-12.
- Method, limits and the full data: [the October 2026 study of the top 1,000 websites](https://www.crawlreadiness.com/research/ai-crawler-access-october-2026).

## What blocking it costs

Nothing in today's answers, and nothing in the readiness score: cohere-training-data-crawler only gathers training data, so blocking it is a policy choice about future models, not about being found. The check lists the block without deducting for it.

## The robots.txt lines

To let it in (a group that names the crawler replaces any wildcard rules for it, so this also lifts a `User-agent: *` block):

```
User-agent: cohere-training-data-crawler
Allow: /
```

To keep it out:

```
User-agent: cohere-training-data-crawler
Disallow: /
```

A rule in robots.txt is a request. Whether this crawler honours it is undocumented. The [robots.txt generator](https://www.crawlreadiness.com/tools/robots-txt) writes a whole file with the same split: block the training crawlers, keep the search and answer crawlers.

## Also from [Cohere](https://www.crawlreadiness.com/crawlers/vendor/cohere)

- [`cohere-ai`](https://www.crawlreadiness.com/crawlers/cohere-ai) · AI answers

## Does cohere-training-data-crawler get through on your site?

[Run the free check](https://www.crawlreadiness.com/): it reads your robots.txt against all 51 crawlers and tells you which ones can read you, and what to change.

---

Source: https://www.crawlreadiness.com/crawlers/cohere-training-data-crawler