[AI crawlers](https://www.crawlreadiness.com/crawlers) · xAI / Grok · AI answers · Counted in the score

# xAI-Grok

xAI-Grok is xAI / Grok's ai answers crawler token. Blocking it costs points in the AI readiness score. 9.3% of the top 1,000 websites with a robots.txt block it.

Another community-listed name for xAI's crawling. xAI documents nothing about it.

## Where this comes from

- **Community-listed; no vendor documentation.** No vendor page names this token.
- **Robots.txt behaviour undocumented.**
- **Counted in the readiness score.** It can keep a site out of AI search or answers, so a block costs points.

## In the top 1,000 websites

- **9.3% block it in robots.txt** (53 of the 569 sites with a robots.txt), as of 2026-10-12.
- Method, limits and the full data: [the October 2026 study of the top 1,000 websites](https://www.crawlreadiness.com/research/ai-crawler-access-october-2026).

## What blocking it costs

xAI / Grok says this token controls more than training, so a block can keep your pages out of its answers or product features. The score counts it, so a block costs points.

## The robots.txt lines

To let it in (a group that names the crawler replaces any wildcard rules for it, so this also lifts a `User-agent: *` block):

```
User-agent: xAI-Grok
Allow: /
```

To keep it out:

```
User-agent: xAI-Grok
Disallow: /
```

A rule in robots.txt is a request. Whether this crawler honours it is undocumented. The [robots.txt generator](https://www.crawlreadiness.com/tools/robots-txt) writes a whole file with the same split: block the training crawlers, keep the search and answer crawlers.

## Also from [xAI / Grok](https://www.crawlreadiness.com/crawlers/vendor/xai-grok)

- [`GrokBot`](https://www.crawlreadiness.com/crawlers/grokbot) · AI answers
- [`Grok-DeepSearch`](https://www.crawlreadiness.com/crawlers/grok-deepsearch) · Fetches when a user asks

## Does xAI-Grok get through on your site?

[Run the free check](https://www.crawlreadiness.com/): it reads your robots.txt against all 51 crawlers and tells you which ones can read you, and what to change.

---

Source: https://www.crawlreadiness.com/crawlers/xai-grok