Skip to content
Control token Run by Google

What is Google-Extended?

Short answer: Google-Extended is a robots.txt token, not a crawler. It controls whether content Google crawls may be used to train future Gemini models and to ground answers in Gemini apps and Vertex AI. Google says it does not affect inclusion or ranking in Google Search, so AI Overviews are not affected.

Google-Extended at a glance

Run by
Google
Type
Control token
Used for
Gemini apps and Vertex AI
robots.txt name
Google-Extended
Follows robots.txt
Control token, never visits
Last checked
18 November 2026
User agent
None of its own Google says Google-Extended has no separate HTTP user agent. Crawling is done with existing Google user agents and the token is read only as a control.

What does Google-Extended do?

Google describes Google-Extended as a standalone product token that web publishers can use to manage whether content Google crawls from their sites may be used for training future generations of Gemini models that power Gemini apps and the Vertex AI API for Gemini. It also covers grounding: using your pages to answer questions in Gemini apps and in Grounding with Google Search on Vertex AI.

A separate token, Google-CloudVertexBot, covers crawls that site owners request when building Vertex AI Agents. Google says it does not affect Google Search or other products.

What happens if you block Google-Extended?

Disallowing Google-Extended keeps your content out of future Gemini training and out of grounded answers in Gemini apps and Vertex AI. Google's October 2026 post notes that sites which specifically allow Google-Extended are eligible to appear in responses and receive clicks via grounding in Gemini apps.

Google says using Google-Extended does not affect a site's inclusion in Search, and Google-Extended is not used as a ranking signal. AI Overviews and AI Mode are part of Search, so they are controlled by Googlebot, not by this token.

How do you set Google-Extended in robots.txt?

Is Google-Extended a crawler? It is a control, not a crawler. Google reads your Google-Extended rules when deciding how it may use content its other crawlers fetched; no request ever arrives from Google-Extended itself.

Add one of these groups to the robots.txt file at the root of your site. Several User-agent lines in a row share the rules below them, and a crawler follows only the most specific group that names it, so repeat any private paths in that group.

Opt out with Google-Extended

User-agent: Google-Extended
Disallow: /

Allow Google-Extended

User-agent: Google-Extended
Allow: /

Should you block Google-Extended?

Our view: If you want Gemini to cite and link to you, leave Google-Extended allowed. Blocking it is a reasonable choice for publishers who license content, but it costs you visibility in Gemini app answers.

What is changing with Google-Extended in 2027?

In October 2026 Google said it plans to support a shared AI training control in robots.txt, based on work in the IETF AI Preferences (AIPREF) working group. If a site uses that shared signal to opt out of training and does not name Google-Extended, Google plans to treat it as if the site had disallowed Google-Extended. Google says company specific controls such as Google-Extended will keep working alongside it.

How do you verify a real Google-Extended request?

There is nothing to verify, because Google-Extended never visits your site. Requests from Google come from its normal crawlers, which you can verify with Google's published IP ranges.

Google-Extended questions

What is the Google-Extended user agent?
Google-Extended has no user agent of its own. Google says Google-Extended has no separate HTTP user agent. Crawling is done with existing Google user agents and the token is read only as a control.
Does Google-Extended follow robots.txt?
It is a control, not a crawler. Google reads your Google-Extended rules when deciding how it may use content its other crawlers fetched; no request ever arrives from Google-Extended itself.
Is Google-Extended a crawler?
No. Google says it has no separate user agent. It is a robots.txt token that controls how Google may use content its existing crawlers fetch.
Does blocking Google-Extended remove me from AI Overviews?
No. Google says Google-Extended does not affect inclusion in Google Search. AI Overviews are part of Search and depend on Googlebot.
Does Google-Extended affect Gemini answers?
Yes. It controls whether your content may be used to train future Gemini models and to ground answers in Gemini apps and Vertex AI.
How do I check whether my site allows Google-Extended?
Run your domain through the free ONEGO AI crawler checker. It reads your robots.txt and shows whether Google-Extended and 15 other AI crawlers are allowed, blocked or partly restricted, and what each rule means.

Crawlers can read you. Does AI recommend you?

See your ONE Score first · 7 day free trial, no card needed · Pay only if you keep tracking