Sites that block AI training crawlers mostly ignore the answer time bots
What happened
Some websites that block AI training crawlers from scraping their content are not blocking answer time bots, which are a type of AI that responds to questions. This is a problem because it allows these answer time bots to continue training on the websites' content, potentially undermining the websites' attempts to block AI scraping. This issue matters because it highlights the cat-and-mouse game between AI developers and website owners trying to control how their content is used.
Why it matters
This means that AI developers may continue to find ways to train their models on website content, even if website owners try to block them, and could lead to further tension between the two groups.
Is this a big deal?
Get stories like this every morning
The day's AI news in plain English, in your inbox at 8:30 IST. Free.
More AI news
Every Way to Get 192GB for Local AI (Without a $9,500 Mac) [video]Industry news · 14 Nov 2026DeepSeek V4 Pro 0813DeepSeek · 14 Nov 2026DeepSeek V4 Pro 0813DeepSeek · 14 Nov 2026Show HN: Portal2Bench – Can AI agents solve Portal 2?Industry news · 14 Nov 2026Anthropic Upgrades Claude Code Projects to Run Parallel AI Workers Hands-FreeAnthropic · 14 Nov 2026This page was collected from sitedex.dev and summarised automatically by AIPulsee's pipeline. See how it runs.