[AI crawlers](https://www.crawlreadiness.com/crawlers) · Google AI · Fetches when a user asks · Counted in the score

# Google-Agent

Google-Agent is Google AI's fetches when a user asks crawler token. Blocking it costs points in the AI readiness score.

Agents hosted on Google's infrastructure that browse the web and take actions when a user asks. Google documents it as a user-agent string, not a robots.txt token, says user-triggered fetchers generally ignore robots.txt, publishes the agents' IP ranges, and is trying Web Bot Auth to identify them.

## Where this comes from

- **Documented by the vendor.** Source: [developers.google.com](https://developers.google.com/crawling/docs/crawlers-fetchers/google-user-triggered-fetchers)
- **May bypass robots.txt for user-requested fetches.**
- **Counted in the readiness score.** It can keep a site out of AI search or answers, so a block costs points.

## In the top 1,000 websites

Not measured in the October 2026 study of the top 1,000 websites.

## What blocking it costs

A rule may not stop it: Google-Agent fetches a page because a person asked about it, and its vendor says such fetches can bypass robots.txt. Where it does honour the rule, the assistant cannot read your page when asked. The score counts it, so a block costs points.

## The robots.txt lines

To let it in (a group that names the crawler replaces any wildcard rules for it, so this also lifts a `User-agent: *` block):

```
User-agent: Google-Agent
Allow: /
```

To keep it out:

```
User-agent: Google-Agent
Disallow: /
```

A rule in robots.txt is a request. Google AI says a fetch a user asked for may go ahead regardless. The [robots.txt generator](https://www.crawlreadiness.com/tools/robots-txt) writes a whole file with the same split: block the training crawlers, keep the search and answer crawlers.

## Also from [Google AI](https://www.crawlreadiness.com/crawlers/vendor/google-ai)

- [`Google-Extended`](https://www.crawlreadiness.com/crawlers/google-extended) · AI answers
- [`GoogleOther`](https://www.crawlreadiness.com/crawlers/googleother) · Research crawl
- [`Gemini-Deep-Research`](https://www.crawlreadiness.com/crawlers/gemini-deep-research) · Fetches when a user asks
- [`Google-GeminiNotebook`](https://www.crawlreadiness.com/crawlers/google-gemininotebook) · Fetches when a user asks
- [`Google-NotebookLM`](https://www.crawlreadiness.com/crawlers/google-notebooklm) · Retired token
- [`Google-CloudVertexBot`](https://www.crawlreadiness.com/crawlers/google-cloudvertexbot) · Only for the site owner
- [`GoogleOther-Image`](https://www.crawlreadiness.com/crawlers/googleother-image) · Research crawl
- [`GoogleOther-Video`](https://www.crawlreadiness.com/crawlers/googleother-video) · Research crawl

## Does Google-Agent get through on your site?

[Run the free check](https://www.crawlreadiness.com/): it reads your robots.txt against all 60 crawlers and tells you which ones can read you, and what to change.

---

Source: https://www.crawlreadiness.com/crawlers/google-agent