[AI crawlers](https://www.crawlreadiness.com/crawlers) · Google AI · Only for the site owner · Not scored

# Google-CloudVertexBot

Google-CloudVertexBot is Google AI's only for the site owner crawler token. Blocking it costs nothing in the AI readiness score. 12.1% of the top 1,000 websites with a robots.txt block it.

Crawls only when a site owner asks for it, to build Vertex AI agents. Google says it has no effect on Google Search or other products.

## Where this comes from

- **Documented by the vendor.** Source: [developers.google.com](https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers)
- **Honours robots.txt.**
- **Listed, not scored.** It cannot keep a site out of today's answers, so the check shows a block without deducting for it.

## In the top 1,000 websites

- **12.1% block it in robots.txt** (69 of the 569 sites with a robots.txt), as of 2026-10-12.
- Method, limits and the full data: [the October 2026 study of the top 1,000 websites](https://www.crawlreadiness.com/research/ai-crawler-access-october-2026).

## What blocking it costs

Nothing: it only crawls when the site owner sets it up. The check lists the block without deducting for it.

## The robots.txt lines

To let it in (a group that names the crawler replaces any wildcard rules for it, so this also lifts a `User-agent: *` block):

```
User-agent: Google-CloudVertexBot
Allow: /
```

To keep it out:

```
User-agent: Google-CloudVertexBot
Disallow: /
```

A rule in robots.txt is a request. Google AI says this crawler honours it. The [robots.txt generator](https://www.crawlreadiness.com/tools/robots-txt) writes a whole file with the same split: block the training crawlers, keep the search and answer crawlers.

## Also from [Google AI](https://www.crawlreadiness.com/crawlers/vendor/google-ai)

- [`Google-Extended`](https://www.crawlreadiness.com/crawlers/google-extended) · AI answers
- [`GoogleOther`](https://www.crawlreadiness.com/crawlers/googleother) · Research crawl
- [`Gemini-Deep-Research`](https://www.crawlreadiness.com/crawlers/gemini-deep-research) · Fetches when a user asks
- [`Google-NotebookLM`](https://www.crawlreadiness.com/crawlers/google-notebooklm) · Fetches when a user asks
- [`GoogleOther-Image`](https://www.crawlreadiness.com/crawlers/googleother-image) · Research crawl
- [`GoogleOther-Video`](https://www.crawlreadiness.com/crawlers/googleother-video) · Research crawl

## Does Google-CloudVertexBot get through on your site?

[Run the free check](https://www.crawlreadiness.com/): it reads your robots.txt against all 51 crawlers and tells you which ones can read you, and what to change.

---

Source: https://www.crawlreadiness.com/crawlers/google-cloudvertexbot