[AI crawlers](https://www.crawlreadiness.com/crawlers) · Kagi · AI search index · Counted in the score

# Kagibot

Kagibot is Kagi's ai search index crawler token. Blocking it costs points in the AI readiness score. 6.9% of the top 1,000 websites with a robots.txt block it.

Builds the index for Kagi's paid, ad-free search. Kagi says it honours rules that name it and otherwise follows a site's Googlebot rules, and publishes its source addresses.

## Where this comes from

- **Documented by the vendor.** Source: [kagi.com](https://kagi.com/bot)
- **Honours robots.txt.** Published addresses: [kagi.com/bot](https://kagi.com/bot).
- **Counted in the readiness score.** It can keep a site out of AI search or answers, so a block costs points.

## In the top 1,000 websites

- **6.9% block it in robots.txt** (39 of the 569 sites with a robots.txt), as of 2026-10-12.
- Method, limits and the full data: [the October 2026 study of the top 1,000 websites](https://www.crawlreadiness.com/research/ai-crawler-access-october-2026).

## What blocking it costs

Your pages leave the search index Kagibot builds, so Kagi's product can no longer find or cite them from its own index. The score counts it among the crawlers that feed AI search and answers, so a block costs points.

## The robots.txt lines

To let it in (a group that names the crawler replaces any wildcard rules for it, so this also lifts a `User-agent: *` block):

```
User-agent: Kagibot
Allow: /
```

To keep it out:

```
User-agent: Kagibot
Disallow: /
```

A rule in robots.txt is a request. Kagi says this crawler honours it. The [robots.txt generator](https://www.crawlreadiness.com/tools/robots-txt) writes a whole file with the same split: block the training crawlers, keep the search and answer crawlers.

## Does Kagibot get through on your site?

[Run the free check](https://www.crawlreadiness.com/): it reads your robots.txt against all 51 crawlers and tells you which ones can read you, and what to change.

---

Source: https://www.crawlreadiness.com/crawlers/kagibot