Google plans to support the shared __usage_ai-training signal for website publishers in early 2027. The proposed robots.txt preference concerns using site content to train generative AI models; it does not itself block crawlers, and Google’s support remains a future plan.
What the proposed signal covers
The IETF draft defines AI training as using retrieved content to change the learned parameters of a generative AI model. Under that definition, exclusively non-generative tasks such as classification, ranking, or scoring are outside the signal’s scope.
The proposed token appears in a User-Agent group in robots.txt, alongside ordinary path rules such as Allow and Disallow. The IETF draft introduces no new robots.txt directives or grammar. It describes a preference about content use, rather than a technical guarantee that content will never be retrieved or used; it also says private agreements may override preferences expressed through the mechanism.
Why it does not block crawling
The draft treats content-use preferences and crawler traffic rules as separate controls. __usage_ai-training is not a crawling directive, so it does not by itself prevent a crawler from visiting a site. A publisher seeking to limit crawling needs crawler-specific robots.txt rules.
That distinction matters in practice: a site can express a preference about training and separately set rules for crawler access. The proposed training signal is also a website-publisher mechanism, not a setting for an individual’s Gemini activity or Google Search history.
Google-Extended and existing controls
John Mueller, Google Search Advocate, said Google plans to treat a shared-token opt-out as disallowing Google-Extended unless a site names Google-Extended separately. He also said existing provider-specific controls would continue to function alongside the shared signal. Those are Google’s reported intentions for its planned support.
Google has said sites that specifically allow Google-Extended are eligible to appear in Gemini Apps responses and receive clicks through grounding. Eligibility does not promise a particular appearance or click.
The proposal’s standards status
The IETF Datatracker identifies version -00 as an individual Internet-Draft, not an IETF-endorsed standard and without formal standing in the IETF standards process. Google's planned support does not change that status.
