AI Although models are becoming increasingly powerful, the problem of hallucination still persists—that is, confidently spouting nonsense. This is why verifying answers remains a crucial step even today. Many people are certainly curious about why language models experience hallucinations. What is the root cause? Earlier— OpenAI An explanation was offered for this, and it was also believed to be solvable, but for now, there’s still a way to go.
Why do language models hallucinate? OpenAI explains, and proposes solutions.
The so-called “hallucinations” refer to when a language model generates answers with great confidence, but the content is entirely wrong—many people have likely encountered this. To address this issue, various prompting techniques have appeared online to effectively reduce the hallucination rate, such as:
- Provide clear background information.
- Require AI to cite sources.
- Break complex problems down into smaller ones.
When getting answers, you can also use the online search feature to cross-reference. Gemini has a “verify reply content” feature, and even lets you ask the AI questions in return.
OpenAI says that language models hallucinate because standard training and evaluation procedures tend to reward “guessing” rather than “acknowledging uncertainty.”
Part of the reason hallucinations persist is that current evaluation methods are designed with the wrong incentives—most of them measure model performance by accuracy rate, which means that when a model encounters an answer it isn’t sure about, it’s encouraged to guess rather than honestly say it doesn’t know.
OpenAI uses exams as an example: if you don’t know the answer, guessing randomly gives you a chance to get it right, thereby improving your accuracy, but leaving it blank always results in zero.
The following are the SimpleQA test scores of the two models:
- gpt-5-thinking-mini: 52% abandonment rate, 22% accuracy rate, 26% error rate.
- o4-mini: abandonment rate 1%, accuracy rate 24%, error rate 75%.
In terms of accuracy alone, o4-mini’s 24% is better, but its error rate (hallucination rate) is terrible, as high as 75%. gpt-5-thinking-mini, by abstaining from many uncertain answers, brings its error rate down to 26%.
Most current leaderboards prioritize accuracy, which leads developers to design models that are more inclined to “guess” rather than “answer honestly.” This is also why, even as models become more powerful, the problem of hallucination persists.
Therefore, to solve this problem, OpenAI believes that mainstream evaluation methods must be redesigned.
There is also a common misunderstanding that improving accuracy will eliminate hallucinations, since 100% accuracy would mean no hallucinations. But in reality, accuracy can never reach 100%, because some problems are inherently unsolvable, much like in real life.
Some people also believe that “to avoid hallucinations, we need large models with higher intelligence.” OpenAI says that small models are actually better at recognizing what they don’t know. For example, when asked a Māori-related question, a small model will directly say “I don’t know.” Compared to a large model that half-understands and has to weigh its confidence, the small model responds more straightforwardly.
The article also mentions that hallucinations originate from the pretraining phase. The model’s task during pretraining is to “predict the next word in large amounts of text,” not to judge whether sentences are true or false. Therefore, due to the lack of “correct/incorrect” labels, the model struggles to distinguish between correct and incorrect content, as it only sees positive examples of language.
If you want to learn more about what OpenAI has published, you can visit the official website.Read full article。
Source: KOCPC Chinese