AI Training Preferences and Publisher Controls Watch
The shared machine-readable AI-training preference signal and the provider controls affecting crawling, training, grounding and AI answer visibility, including the distinction between a proposed standard, provider implementation and publisher-level crawler settings.
Series updated · 2 published updates
The next developments that matter
- Revisions and working-group decisions for the IETF AI Preferences drafts, including the current draft expiry and any move toward an adopted RFC.
- OpenAI implementation details and how the shared token interacts with OAI-SearchBot, GPTBot and user-triggered ChatGPT-User retrieval.
- Google’s 2027 rollout and any updates to the interaction between __usage_ai-training, Google-Extended, Gemini training and Gemini grounding.
- Microsoft’s 1 February 2027 effective date, page-level grounding controls and 1 March transition steps for publishers still using NoArchive or NoCache.
- Anthropic’s implementation across Claude services and any changes to its crawler documentation.
- Official implementation statements or verifiable technical tests from other providers; industry commentary alone does not establish adoption.
- The distinction between a publisher’s own crawler directives and the proposed AI-training preference token: they address different questions and can affect crawling, model training and answer retrieval in different ways.
This timeline highlights developments that materially change the story; it is not a live news feed.
The series timeline
Oldest to newest- 01
AI Training and robots.txt: What the New Shared Signal Means
OpenAI, Google, Microsoft and Anthropic have moved a shared AI-training preference signal forward. Here is what the proposal says, where implementation stands and what website owners should check before changing robots.txt.
Read this update → - 02
One AI Training Signal, Different Grounding Rules: What Website Owners Need to Know
A shared robots.txt training preference is emerging, but Google, Microsoft, OpenAI and Anthropic still handle crawling and AI answer visibility differently.
Read this update →
Reading this timeline
Entries are arranged chronologically. Each article has its own publication date and sources, so readers can see what was known at each stage and how the public record developed.