Making it easier for websites to express AI training preferences
What happened
OpenAI announced it will honor a new common signal placed in a website’s robots.txt file that lets operators state their preferences about using site content for foundational generative AI training. The voluntary signal lets publishers communicate the same choice to multiple AI providers at once, replacing the need to write separate rules for each crawler. OpenAI says it will implement the signal in the coming months and will keep its existing crawler controls in robots.txt. The effort builds on work by the IETF’s AI Preferences working group and encourages other AI providers to adopt the same approach.
Why it matters
For everyday web users, this means website owners can more easily tell AI services whether their content may be used for training, potentially reducing unwanted data harvesting. It also signals a move toward clearer, shared standards for how online content is handled by AI systems.
Is this a big deal?
Get stories like this every morning
The day's AI news in plain English, in your inbox at 8:30 IST. Free.
More from OpenAI
Introducing ChatGPT WalletOpenAI · 13 Nov 2026Shield helps technicians hand off legacy portal work with OpenAIOpenAI · 13 Nov 2026OpenAI launches ChatGPT Wallet, AI agents pay for purchaseOpenAI · 13 Nov 2026OpenAI disbands its robotics team as its general models start running robotsOpenAI · 13 Nov 2026Yang-Mills, OpenAI proof, and the mass gap problemOpenAI · 13 Nov 2026This page was collected from openai.com and summarised automatically by AIPulsee's pipeline. See how it runs.