High impact

Calling the AI bluff: Adding "Do not guess" cut made-up claims from 71% to 20%

What happened

A benchmark tested 16 web‑extraction models on 42 page pairs containing hidden fields, instructing them to return null when a value was missing and not to guess. Without the “Do not guess” instruction, the models fabricated answers for 405 of 573 missing fields (70.7%); with the instruction they fabricated only 116 of 574 (20.2%). Firecrawl performed worst, making up 24 of 36 missing fields, while a plain fetch plus GPT‑6 Luna fabricated just 5 of 36. A cheap checker model (GPT‑6 Luna) could verify answers for a few cents, catching most fabricated values.

Why it matters

For users hiring AI‑powered data extraction services, the simple “Do not guess” prompt dramatically reduces false data, and inexpensive verification can further ensure accuracy. This lowers the risk of costly errors when agents rely on scraped information.

Reality check

An automated, sceptical read of the source

Mostly marketingIncrementalNotableGenuine breakthrough

Incremental. The result shows a modest but practical improvement from a prompt tweak and cheap checking, rather than a fundamentally new technology.

Who it affectsDevelopersBusinessesEveryday users

What to watchWatch for broader adoption of prompt‑based safeguards and low‑cost verification tools in commercial web‑scraping services.

Do you agree with this rating?Agree Disagree

More AI news

Show HN: New Cartesia Text-to-Speech ModelIndustry news · 13 Nov 2026Chicago schools to Google – we'd rather lose Search than be forced to accept AIIndustry news · 13 Nov 2026Introducing ChatGPT WalletOpenAI · 13 Nov 2026Shield helps technicians hand off legacy portal work with OpenAIOpenAI · 13 Nov 2026How AI Is Helping Care Teams Get Patients the Right Care, SoonerNVIDIA · 13 Nov 2026

This page was collected from earnanhonestdollar.com and summarised automatically by AIPulsee's pipeline. See how it runs.