Using LLM for confidence scoring is a bad idea
What happened
The article argues that using large language models (LLMs) to produce confidence scores is unreliable. It explains confidence by comparing it to weather forecasting, where true confidence comes from data and calibrated predictions over many instances. The author notes that a forecaster (or LLM) can be calibrated yet uninformative if it ignores current information. The piece emphasizes that LLMs lack the underlying information and statistical grounding needed to generate trustworthy probabilities, making their confidence numbers essentially opinions rather than calibrated measures.
Why it matters
For everyday readers, this means that AI-generated confidence numbers should not be taken at face value when making decisions, as they may be misleading. Trusting such scores could lead to overconfidence in AI outputs and poor outcomes.
Reality check
An automated, sceptical read of the sourceIncremental. The claim challenges a common practice but does not introduce a new technology, merely urging caution about existing LLM behavior.
Is this a big deal?
Get stories like this every morning
The day's AI news in plain English, in your inbox at 8:30 IST. Free.
More AI news
AI unlocks Gödel's notebooks, said to contain hundreds of proved theoremsIndustry news · 13 Nov 2026Gemini 4 Hydrogen and OrchestratorIndustry news · 12 Nov 2026Gemini 4 HydrogenIndustry news · 12 Nov 2026I'm making LLMs work on small and old devicesIndustry news · 12 Nov 2026United Airlines bans AI agents from booking flights on behalf of travelersIndustry news · 11 Nov 2026This page was collected from rumple.github.io and summarised automatically by AIPulsee's pipeline. See how it runs.