Google's AI opt-out is not a crawler block

Google's new generative AI control is a Search Console toggle that decides whether your site can appear in, and help ground, answers in AI Overviews and AI Mode. It is an eligibility switch. It is not a crawler block, it is not a robots.txt rule, and it does not touch your ranking in ordinary search results.

That distinction is the whole story, because for two years the only tools site owners had were crawler and snippet directives that were never designed for this job. Teams have been reaching for Google-Extended expecting it to remove them from AI Overviews. It never did.

#Why the control exists now

This did not arrive because Google changed its mind about publisher controls. It arrived because a regulator ordered it.

On 3 June 2026 the UK's Competition and Markets Authority imposed a conduct requirement on Google following its strategic market status designation in general search. The CMA describes the result as a world first: publishers get effective tools to prevent their content being used to power AI features in search, such as AI Overviews (CMA). The same requirement covers opting out of fine-tuning, and obliges Google to attribute publisher content with clear links inside AI-generated results (CMA).

Google has nine months to implement the full package, and the CMA said it expects important parts to land well before that deadline (CMA). That timing explains the rollout shape. Google is testing the toggle with a subset of site owners in the UK first, with global expansion promised after testing (Google).

If you are outside the UK, you cannot flip this switch yet. You can still decide now what you would do with it, and you should, because the reasoning takes longer than the click.

#What the toggle actually controls

Google's wording is narrow and worth reading literally. Site owners can decide whether they want their site to appear in and help ground responses in generative AI Search features like AI Overviews, AI Mode, or AI Overviews in Discover (Google).

Two verbs, two effects. "Appear in" is citation: your link showing up in the answer. "Help ground" is retrieval: your content being pulled in as evidence the model writes from. The toggle collapses both. You cannot keep the grounding and drop the citation, or the reverse.

Google is explicit about the cost and about the boundary. Sites that opt out will not receive traffic or impressions from generative AI features, and the setting will not be used as a ranking signal for search results outside those features (Google).

Take that second clause at face value, but read it precisely. It promises that opting out will not demote you in the ten blue links. It does not promise the ten blue links will carry the volume they used to.

#Three levers, three different jobs

The reason so many teams got this wrong is that the existing controls look adjacent and are not.

robots.txt for Googlebot governs crawling. Google states plainly that robots.txt directives for Googlebot are the control for managing how sites are crawled for Search (Google). Block Googlebot and you leave Search entirely, AI features included, along with everything else.

nosnippet, data-nosnippet, and max-snippet govern what can be shown. These are what Google points to for limiting content appearance in AI features (Google). They work, and they are blunt in a specific way: the same directive that withholds text from an AI Overview also withholds your regular search snippet. data-nosnippet at least lets you fence off sections of a page rather than the whole thing.

Google-Extended governs neither. It applies to training and grounding in Google's other systems, outside Search (Google). A site that added Google-Extended to robots.txt in 2024 believing it had removed itself from AI Overviews was appearing in them the entire time. We wrote about the general shape of this confusion in what partial AI bot access actually means.

The new toggle is the first control that separates AI-feature eligibility from crawling and from snippet display. That is genuinely new, and it is the argument for waiting for it rather than reaching for a snippet directive today.

#Opting out does not reduce the AI answer

The mental model most teams bring to this is a tap: turn it off and less of my content flows into AI. The real mechanic is a seat at a table that stays fully occupied.

Search Engine Land put the point directly when the controls were announced: they do not turn off AI Overviews, stop people using AI Mode, or slow AI adoption (Search Engine Land). The query still gets answered. The answer still gets synthesized. The sources are simply drawn from whoever remains eligible.

Scale is what makes this asymmetric. AI Overviews has more than 2.5 billion monthly active users and AI Mode has passed one billion monthly users (Google). Withdrawing from surfaces at that scale is not a protest that costs Google something measurable. It is a transfer of citation share to the sites that stay in.

There are business models where that transfer is worth it. Subscription publishers whose content is the product, sites whose value is locked behind a paywall that AI summarization erodes, licensing negotiations where withdrawal is real bargaining power under the CMA's attribution and opt-out regime. For most B2B SaaS, where the page exists to be found and the AI answer is the discovery layer, it is a straightforward loss. We laid out that decision framework in block or allow AI crawlers.

#The part that gets missed: you also go dark on measurement

Google shipped the reporting alongside the control, and the two interact in a way worth planning around.

The new Search Generative AI performance reports break out impressions within generative AI features, with dimensions for pages, countries, devices, and dates at hourly, daily, weekly, and monthly granularity. They do not include click data, and like the toggle they started with a subset of UK site owners (Search Engine Land).

Now put the two together. Opting out means no impressions from generative AI features (Google). Impressions are what the report counts. So the moment you opt out, the only first-party data Google gives you about your AI-feature visibility goes to zero, and it goes to zero because you switched it off, not because the market moved.

You lose the ability to answer the question you opted out to investigate. There is no A/B here and no control group. Re-enabling and waiting for data to rebuild is the only way back, and you will be comparing across a changed period.

#The sequence that makes sense

Measure before you decide. Get a baseline of AI-feature impressions and which URLs earn them, ideally over a period long enough to survive normal variance rather than a single week. Our note on why AI visibility is a distribution covers how much sampling that actually takes.

Then check what you already have switched on. Audit robots.txt and your snippet directives for controls added under the old assumptions, particularly Google-Extended entries added to stop AI Overviews and nosnippet rules that are quietly costing you regular snippets too.

Then decide with the number in front of you, not the anxiety. The toggle is reversible in principle, but the measurement gap it creates is not, and the seat you vacate gets filled the same day.