Four AI companies back one publisher opt-out for training

OpenAI, Google, Microsoft, and Anthropic say they will recognize a shared instruction in robots.txt, the small public file websites use to communicate rules to automated visitors. Instead of maintaining a separate block for every company’s bot, a publisher will be able to mark all or part of a site as unavailable for training generative AI models through one common signal. That matters because publisher controls have become a patchwork as disputes over the value and use of online content have intensified. It is not a universal stop button: participation is voluntary, the underlying proposal is still a draft, and the signal applies to future model training—not by itself to crawling, search visibility, or AI answers based on live web pages. Anthropic has begun implementation, while the other companies plan support in coming months or in 2027.