GPT-5 Hallucination Rates: Benchmarks Tell Conflicting Stories

A record score with a catch

OpenAI's GPT-5 family has been promoted as a step change in reliability. Independent testing from Artificial Analysis complicates that picture. On the firm's private AA-Omniscience benchmark, GPT-5.5 at its highest reasoning setting (xhigh) posted the best accuracy the test has recorded: 57% 2. It also posted a hallucination rate of 86%, far worse than Anthropic's Opus 4.7 (max) at 36% and Google's Gemini 3.1 Pro Preview at 50% 2.

Those two numbers measure different things. AA-Omniscience rewards a model for recalling facts across diverse topics but penalizes it for making things up 2. Artificial Analysis says the high hallucination figure means GPT-5.5 is more likely to answer a question even when it does not actually "know" the answer 2. In plain terms, the model knows more than its rivals, but when it reaches the edge of its knowledge it tends to bluff rather than say it isn't sure.

The firm credited GPT-5.5's 14-point jump on AA-Omniscience over GPT-5.4 (xhigh) mainly to better knowledge, with only a modest improvement in hallucination 2. GPT-5.5 also leads Artificial Analysis's GDPval-AA leaderboard with an Elo of 1785 and gained 7 points on τ²-Bench Telecom, a customer-service agent test 2. By most capability measures, it is a frontier-leading model. Calibration is the exception.

Where the low-hallucination story came from

The reputation for reliability has some support. In OpenAI's own testing, researchers removed images from multimodal prompts. GPT-5 confidently described the missing images only 9% of the time, compared with 86.7% for the earlier o3 model 3. That is a large improvement, but it covers a narrow failure mode. It tests whether a model invents details about an input that isn't there, not whether it admits ignorance on hard factual questions.

Summarization benchmarks have also helped GPT-5. On Vectara's original hallucination leaderboard, built on roughly 1,000 documents, GPT-5 (high) scored a 1.4% hallucination rate, or 98.6% factual consistency 1. On Vectara's newer and larger dataset of 7,700 articles, the same model's rate rose to 15.1% 1. The model did not change; the test got harder. That more than tenfold gap shows how much a headline hallucination figure depends on what is being measured.

The broader leaderboard

The October AA-Omniscience rankings show how differently models handle the tradeoff between accuracy and abstention. Cohere's Command A+ has the lowest hallucination rate at 14%, but only 9% accuracy 1. It avoids errors mostly by declining to answer. Google's not-yet-public Gemini 4 Argon (high) stands out by pairing a 15.1% hallucination rate with 49.9% accuracy 1. xAI's Grok 4.3 (medium) and Grok 4.20 (Reasoning) both sit below 18% hallucination with accuracy in the high 20s 1. Further down, Anthropic's Claude 4.5 Haiku comes in at 25.7%, and Meta's Muse Spark variants land in the low 30s 1.

Compared with that field, GPT-5.5's 86% stands out. The comparison carries a caveat, though. The October table and Artificial Analysis's GPT-5.5 write-up appear to cover somewhat different model generations and snapshots, so the figures are best read as indicating a pattern rather than as one unified ranking.

Why this matters

These sources do not contradict each other so much as measure different behaviors. OpenAI's 9% figure, Vectara's 1.4% and 15.1%, and Artificial Analysis's 86% can all be accurate at once 123. The problem comes when one flattering number is presented as a general verdict on a model's honesty.

For anyone deploying these systems, the AA-Omniscience result is arguably the most relevant to real use. Chatbots routinely face questions outside their knowledge. A model that recalls more facts but rarely admits uncertainty can be more dangerous than a less knowledgeable one that knows when to stop, because its errors come with the same confidence as its correct answers.

The takeaway

GPT-5.5 is, by Artificial Analysis's measures, the most knowledgeable model available and the leader on several practical benchmarks 2. It is not a low-hallucination model in the sense that matters most for open-ended factual questions. OpenAI's reliability gains are real but narrow, and Vectara's results show how much they shrink on harder material 13. Rivals such as Anthropic, and Google's upcoming Gemini 4 Argon, currently appear better calibrated 12. Until a model leads on both accuracy and willingness to abstain, readers should treat any single hallucination statistic with caution.