Google, OpenAI, Anthropic and Microsoft Back One Robots.txt Opt-Out for AI Training as Gemini Opens to Outside Apps

Four of the largest AI developers have lined up behind a single way for websites to say no to AI training. On October 30, Google, OpenAI, Anthropic and Microsoft backed a robots.txt token, `usageai-training`, defined in an IETF individual Internet-Draft written by Kevin Kelley of Anthropic. It is a proposal rather than an adopted standard: the IETF has not endorsed it, and the draft expires May 3, 2027.

Adoption is staggered. Anthropic has already started implementing it, OpenAI says it will follow in the coming months, Google expects to support it in early 2027 through its Google-Extended crawler token, and Microsoft says its support begins February 1, 2027. The signal covers training only, does not affect search indexing, and cannot undo scraping that has already happened. Meta, Amazon and Apple are not among the four.

The opt-out landed alongside a run of Gemini product news for GOOGL. On October 28 Google opened its Connected Apps Platform in preview, letting outside developers build apps that run inside AI Mode in Search and the Gemini app through an MCP server or Android AppFunctions; the preview is US-only, and Google says more than 50 Connected Apps are live across the two products. A day later Gemini gained the ability to edit generated documents, PDFs, presentations and spreadsheets directly from the chat window.

For publishers, one shared signal is simpler than tracking crawler names company by company, but it carries no legal force and leaves out several large AI players. For GOOGL and MSFT, the commitments matter for how much publisher content stays available for training and, in Google's case, grounding. The details to watch are OpenAI's start date, how many sites adopt the token, and whether the IETF working group takes the draft up.