How to get found in Gemini: the 2 robots.txt names you control
Short answer: Nobody can promise that you will be found in Gemini, and Google's pages describe no way to be named there. What you control are two names in robots.txt. Googlebot governs crawling for Google Search, including AI Overviews and AI Mode. Google-Extended governs whether Google may use your content to train future Gemini models and for grounding in Gemini Apps. Keeping both open removes the two blockers Google documents; it promises no mention. Google does not count switches; the 2 is our grouping.
What Google documents about getting found in Gemini
Start with the limit. The two Google pages we rely on describe controls, not selection. The list of Google's common crawlers says what the Google-Extended token covers. The page AI features and your website explains how AI Overviews and AI Mode in Search treat your pages. Neither says how Gemini chooses which pages to use in an answer, and we know of no Google source that does. So this article promises no mention. It also says nothing about what else Gemini may draw on: that is not documented there, and we do not guess.
We fetched both pages on 4 November 2026. The crawler list shows "Last updated 2026-10-30", the AI features page "Last updated 2025-12-10". Everything we say about Google comes from these two pages, and we mark our own readings.
One term needs a plain explanation. Grounding is defined on the crawler page as "providing content from the Google Search index to the model at prompt time to improve factuality and relevancy". In short: when someone asks a question, the model can be handed content from Google's Search index. Whether a given Gemini answer does that is not on the page.
What you can control is two names in robots.txt, and that is where the 2 in the title comes from. Google does not count switches; the number is our grouping of the names its pages give you. The preview controls nosnippet, data-nosnippet, max-snippet and noindex are a different kind of tool, and we place them under switch 1.
Switch 1: Googlebot, the door into Google Search and its AI features
According to the AI features page, AI is built into Search, which is why "robots.txt directives for Googlebot is the control" for how your site is crawled for Search. The crawler list adds that crawling preferences for the Googlebot token "affect Google Search (including Discover and all Google Search features)", along with other products such as Google Images and Google News.
AI Overviews and AI Mode have no gate of their own. To be eligible as a supporting link, Google says, a page must be "indexed and eligible to be shown in Google Search with a snippet", and there are no additional technical requirements. Our article on AI Overviews and AI Mode walks through that page.
Why does this switch matter for Gemini? Our reading, not a Google statement: grounding draws on the Search index, so a page that Googlebot cannot crawl gives that index nothing to offer. That is the reason to check Googlebot first, before anything with a Gemini name on it.
The dial on this switch: preview controls. To limit what Search shows from your pages, Google names nosnippet, data-nosnippet, max-snippet and noindex. They limit what Search shows from a page; they do not decide whether Googlebot may crawl it. To test them, the page points to the URL Inspection tool in Search Console, which shows the HTML Googlebot received while crawling the page.
Switch 2: Google-Extended, the switch for Gemini training and grounding
Google describes Google-Extended as a standalone product token with which publishers manage whether content Google crawls from their sites may be used for two things: training future generations of the Gemini models that power Gemini Apps and the Vertex AI API for Gemini, and grounding in Gemini Apps and in Grounding with Google Search on Vertex AI.
Three details from the page:
- It is a token, not a bot. Google-Extended has no separate HTTP user agent string. Crawling is done with Google's existing user agent strings, and the robots.txt token only serves as a control. Our reading: no request in your server log carries the name Google-Extended, so look in robots.txt, not in the log.
- It does not touch Search. "Google-Extended does not impact a site's inclusion in Google Search nor is it used as a ranking signal in Google Search."
- It is for "other systems". The AI features page sends you to Google-Extended only to limit "AI training and grounding in some of Google's other systems". For the AI features inside Search it names Googlebot rules and the preview controls.
Whether to allow or block is your decision, and neither page recommends either. Allowing means Google may use your content for those two purposes; it does not mean Gemini will use or name a page. Blocking tells Google not to use the content for them; the page describes no other effect.
The crawler list also carries a note: Google intends to support the AIPREF shared machine-readable training preference signal in early 2027. Today, Google-Extended is the control that page documents.
Which switch controls what
| Your goal | What to look at | What Google's pages say |
|---|---|---|
| Let Google crawl your pages for Search, AI Overviews and AI Mode | Googlebot in robots.txt (switch 1) | Googlebot rules are the control over crawling for Search; a page must be indexed and eligible to show with a snippet |
| Limit what Search shows from a page | nosnippet, data-nosnippet, max-snippet, noindex (the dial on switch 1) | Named for limiting the information shown in Search |
| Allow or refuse training of future Gemini models | Google-Extended in robots.txt (switch 2) | A standalone token with no effect on inclusion in Search |
| Allow or refuse grounding in Gemini Apps | Google-Extended in robots.txt (switch 2) | The same token as for training |
How to set both switches in robots.txt
# Switch 1: Googlebot has no group of its own, so the "*" group applies to it
User-agent: *
Disallow: /api/
# Switch 2: Google-Extended gets its own group and refuses everything
User-agent: Google-Extended
Disallow: /This example is a choice, not a recommendation: it keeps Google Search open and refuses Gemini training and grounding. If you want Google to be allowed to use your content, leave the Google-Extended group out; with no group of its own it follows the * group under RFC 9309 (Google’s crawler page does not state this for Google-Extended explicitly).
RFC 9309 defines the rule behind this: a crawler follows the group that matches its name and only without a match the * group. Two consequences:
- If you add a group for Googlebot, repeat every rule you still need in it. The
*group no longer applies to Googlebot. - A
Disallow: /under*closes both switches. It is often a leftover from a staging site. It holds for Googlebot and Google-Extended unless one of them has a group of its own.
The same standard says robots.txt is "not a substitute for valid content security measures". It is a request, not a lock.
Our own file is an example, not a recommendation. On 4 November 2026 we fetched https://seogrinder.com/robots.txt with curl: one shared group with * and nine AI user agents, Google-Extended among them, allowing everything except /api/. Googlebot has no group of its own and follows *, so both switches are open there. That is the site owner's decision; yours can differ.
To check your own file, run curl -sS https://example.com/robots.txt and look for Googlebot, Google-Extended and any User-agent: * group. Then allow time. Google says crawling can take "anywhere from several days to several months"; that sentence is about preview controls. The crawler list gives no time for a Google-Extended change, so we give none.
How SEOgrinder helps
SEOgrinder is free and needs no account, and we build it. Enter an address, start the analysis and open the report. Two places matter here:
- AI readiness tab, "Checks for AI agents": the point "Steer AI bots" counts the AI crawlers that have an explicit rule in robots.txt, how many are allowed and how many are blocked, and names the blocked ones. It knows Google-Extended by name and counts it as a training crawler. The point is met when training and search each have an explicit rule. Googlebot is not part of this point.
- Tasks tab: the finding "robots.txt blocks this URL" appears when a Disallow rule keeps Googlebot from the page you scanned, and "Page is excluded from Google" when a noindex directive is set.
Press Copy agent prompt at a point to get an instruction for your coding assistant, and Check again after a change. SEOgrinder does not read your server log and does not measure whether Gemini names your site. It checks the technical basis.
Common mistakes
- Blocking Google-Extended to leave AI Overviews. For the AI features in Search, Google's page names Googlebot rules and preview controls. It says Google-Extended does not impact inclusion in Google Search.
- Looking for Google-Extended in the server log. It has no separate user agent string.
- A Googlebot group that drops your other rules. A name with its own group ignores the
*group. - A
Disallow: /from a staging site still in the file. It closes both switches at once. - Reading "allowed" as "named". Allowing removes a blocker. It promises no mention.
- Expecting an instant effect. Google gives no time for Google-Extended, and days to months for preview controls.
- Treating robots.txt as protection. The standard calls it no substitute for content security.
FAQ
Can I make Gemini name my site?
No. Nobody can force it, and neither Google page describes a setting that causes it. You can remove blockers so that Google may crawl and use your content. Whether a Gemini answer names it is not documented.
Does blocking Google-Extended hurt my rankings or remove me from AI Overviews?
Google says Google-Extended does not impact a site's inclusion in Google Search and is not used as a ranking signal there. For AI Overviews and AI Mode, the AI features page points to Googlebot rules and preview controls.
Do I need an llms.txt or special markup for Gemini?
For AI Overviews and AI Mode, Google says you do not need new machine-readable files, AI text files or markup, and no special schema.org structured data. About Gemini Apps, that page says nothing. We know of no source that proves a benefit of llms.txt.
Does SEOgrinder show whether Gemini names my site?
No. It checks whether robots.txt blocks Googlebot from the page you scanned and whether it has a rule for Google-Extended. Tools that track mentions do a different job.
Your next step
Open your robots.txt and look for Googlebot, Google-Extended and any User-agent: * group. Make sure no leftover rule blocks Googlebot, and decide Google-Extended on purpose. Then run the free scan in SEOgrinder, open the AI readiness tab and the Tasks tab, and fix what is open. Our article on robots.txt for AI crawlers covers the bots of OpenAI, Anthropic and Perplexity.