Medium impact
Only 8.9% of sites block AI crawlers, but 94.8% are never cited in AI answers
What happened
A study by Website Auditor found that most websites don't block AI crawlers, but 94.8% of websites are not being crawled by AI systems, which means AI models are not getting accurate information to provide answers. This is a problem because AI relies on data to learn and provide accurate information. As a result, AI answers may be incomplete or inaccurate.
Why it matters
This highlights the need for better AI data collection methods and more website owners to block AI crawlers to ensure AI models have access to accurate and up-to-date information.
Is this a big deal?
Get stories like this every morning
The day's AI news in plain English, in your inbox at 8:30 IST. Free.
More AI news
Every Way to Get 192GB for Local AI (Without a $9,500 Mac) [video]Industry news · 14 Nov 2026DeepSeek V4 Pro 0813DeepSeek · 14 Nov 2026DeepSeek V4 Pro 0813DeepSeek · 14 Nov 2026Show HN: Portal2Bench – Can AI agents solve Portal 2?Industry news · 14 Nov 2026Anthropic Upgrades Claude Code Projects to Run Parallel AI Workers Hands-FreeAnthropic · 14 Nov 2026This page was collected from website-auditor.io and summarised automatically by AIPulsee's pipeline. See how it runs.