“Be brief” — a sure way to make a chatbot make mistakes more often
It turns out that when we ask a chatbot to give a brief answer, this can significantly increase the likelihood of generating false information. Giskard, a French company engaged in testing artificial intelligence systems, conducted a detailed study on this topic. Scientists have established that requests for short answers, especially on ambiguous topics, can substantially reduce the factual accuracy of artificial intelligence models’ responses.
As researchers note, even simple changes in instructions to the system can radically affect the model’s tendency to hallucinate. That is, to create information that does not correspond to reality. This discovery has serious implications for practical application, since many applications are specifically configured for brief answers in order to reduce data usage, improve speed, and reduce costs.
The problem of hallucinations remains one of the most difficult to solve in the field of artificial intelligence. Even the most modern models sometimes produce made-up information. This is a feature of their probabilistic nature. And interestingly, newer models based on reasoning algorithms, such as OpenAI o3, hallucinate even more often than their predecessors.
In its study, Giskard identified certain queries that exacerbate the hallucination problem. For example, vague questions or those containing erroneous premises with a requirement for a brief answer.
Why does this happen? According to Giskard researchers, when models are not allowed to answer in detail, they simply don’t have the “space.” To acknowledge false premises and point out errors. In other words, more elaborate explanations are required for convincing refutation.
I think there is now a certain conflict between optimization for user experience and factual accuracy. And it turns out that when models are forced to be brief, they consistently choose brevity at the expense of accuracy.
