What Are AI Hallucinations?

Understanding AI Hallucinations

An AI hallucination (or artificial hallucination) is a phenomenon where a generative artificial intelligence model produces an output that is incorrect, nonsensical, ungrounded, or entirely fabricated, while presenting it with a high degree of confidence and plausibility. In natural language processing, this typically manifests as an AI generating false historical dates, inventing non-existent academic papers and legal citations, or misrepresenting facts found within an input prompt.

The term is borrowed metaphorically from human psychology, where a hallucination refers to a perception in the absence of an external stimulus. In generative AI, however, the model is not conscious or experiencing sensory illusions; rather, it is executing statistical pattern completion over probabilistic distributions without an inherent mechanism for understanding objective reality or verifying truth.

Taxonomy and Categorization

Researchers broadly classify hallucinations in large language models (LLMs) and multimodal systems along two major dimensions: factuality and faithfulness.

                     ┌───────────────────────────────┐
                     │       AI Hallucinations       │
                     └───────────────┬───────────────┘
                                     │
             ┌───────────────────────┴───────────────────────┐
             ▼                                               ▼
┌─────────────────────────┐                     ┌─────────────────────────┐
│ Factuality Hallucination│                     │Faithfulness Hallucination│
├─────────────────────────┤                     ├─────────────────────────┤
│ • Factual Inconsistency │                     │ • Instruction Inconsistency│
│ • Factual Fabrication   │                     │ • Context Inconsistency │
│ (Contradicts real-world │                     │ • Logical Inconsistency │
│  knowledge)             │                     │ (Contradicts user input │
│                         │                     │  or context)            │
└─────────────────────────┘                     └─────────────────────────┘

1. Factuality Hallucinations

A factuality hallucination occurs when the model's generated text contradicts established, verifiable world knowledge or invents claims that cannot be grounded in factual reality.

• Factual Inconsistency: The model makes a statement that directly conflicts with known facts (e.g., stating that “Neil Armstrong set foot on Mars in 1969” instead of the Moon).
• Factual Fabrication: The model invents non-existent entities, events, or artifacts from scratch (e.g., fabricating a court case complete with real judge names, fake case numbers, and fictitious legal precedents).

2. Faithfulness Hallucinations

A faithfulness hallucination occurs when the model's output deviates from, contradicts, or misinterprets the specific context, constraints, or source documents provided directly by the user in the prompt.

• Context Inconsistency: When asked to summarize a specific 500-word article, the model introduces external claims or alters the numbers given in the text.
• Instruction Inconsistency: The model ignores or contradicts explicit negative constraints (e.g., asked to produce a response without mentioning dates, but including them anyway).
• Logical Inconsistency: The model provides a line of reasoning where the conclusion directly contradicts the premises it laid out in the preceding sentences.

Root Causes of Hallucinations

Hallucinations are not simple programming bugs that can be patched with standard software debugging; they arise from the fundamental architecture, training objectives, and data foundations of modern deep learning models.

Statistical Prediction vs. World Understanding

Autoregressive language models operate by predicting the next most likely token (a word or sub-word unit) given a sequence of preceding tokens:

The objective of the model is to maximize the statistical likelihood of the sequence based on training patterns, not to evaluate factual veracity. If a syntactically elegant lie has a higher aggregate conditional probability in the model's parameter space than an awkward truth, the model will naturally output the falsehood.

Data-Level Flaws

Models inherit the biases, inaccuracies, and gaps present in their massive web-scale training datasets:

• Noise and Misinformation: Web crawls contain satire, outdated facts, conspiracy theories, and contradictory perspectives that models absorb uniformly.
• Knowledge Decay: Information that was correct at the time of training data collection may become false over time (temporal drift).
• Frequency Bias: Entities or associations that appear frequently in text corpora dominate generation over rarer, more accurate nuances.

Alignment and Training Dynamics

• Over-Optimization via Reinforcement Learning (RLHF): Training models with Reinforcement Learning from Human Feedback can incentivize the model to sound convincing, authoritative, and helpful rather than admitting uncertainty, as human evaluators frequently rate confident-sounding answers higher than non-committal or cautious ones.
• Attention Degradation: In long contexts, transformer attention mechanisms can suffer from "lost in the middle" phenomena, leading the model to overlook key source constraints and revert to ungrounded parametric memory.

Real-World Impact and High-Stakes Risks

While hallucinations in creative writing or brainstorming may appear harmless, their occurrence in mission-critical domains carries severe legal, financial, and safety consequences. [3](#source-3 "Hallucination (

What an AI Hallucination Actually Is

An AI hallucination is a confident, fluent output from an AI system that contains false, fabricated, or unsupported information presented as if it were fact. The term is most often applied to large language models (LLMs) such as chatbots and writing assistants, but it also covers image models that invent details, speech-recognition systems that insert words nobody said, and vision systems that "see" objects that are not in a picture. The defining feature is not simply that the model is wrong — it is that the output is plausible: well-formed, stylistically confident, and often indistinguishable in tone from the model's correct answers. What Are AI Hallucinations? | IBM Hallucination (artificial intelligence) - Wikipedia

A useful mental model: a language model is not a database that looks up facts and returns them. It is a statistical system trained to continue text in a way that resembles its training data. When the pattern it has learned points toward a fluent continuation but the underlying fact is missing, ambiguous, or was never in the training data at all, the model still produces a continuation. That continuation can be a real citation or an invented one, and the model has no built-in signal telling it — or you — which one it produced. Researchers describe hallucinated text as output that is fluent and coherent yet factually incorrect, logically inconsistent, or entirely fabricated. Survey and analysis of hallucinations in large language models - PMC

The terminology is genuinely disputed

"Hallucination" became the popular label, but it is a metaphor borrowed from clinical psychiatry, and a number of researchers object to it. In medicine, a hallucination is a perception without an external stimulus, associated with conditions such as schizophrenia or Parkinson's disease; applying that word to a text generator implies a perceiving mind where there is none. Critics have proposed confabulation (the neurological phenomenon of filling memory gaps with plausible invented detail) as a closer analogy, since it emphasises fluent gap-filling rather than false perception. Others prefer blunter framings, arguing that the systems are indifferent to truth rather than mistaken about it. These alternatives — confabulation, fabrication, and less formal terms — appear alongside "hallucination" in the literature, and no single label is universally accepted. Hallucination (artificial intelligence) - Wikipedia Hallucination or Confabulation? Neuroanatomy as metaphor in ...

Types of Hallucination

The most established distinction in the natural language generation literature separates hallucinations by what they contradict. An intrinsic hallucination conflicts with the source material the model was given — a summary that states the opposite of what the article said. An extrinsic hallucination adds content that the source neither supports nor contradicts, so it cannot be verified from the input at all. This maps onto a second pair of concepts: faithfulness (does the output stay true to the provided source?) and factuality (is the output true about the world?). A response can be faithful to a flawed document while being factually wrong, or factually correct while being unfaithful to the text it was supposed to summarise. Survey of Hallucination in Natural Language Generation

Why Models Hallucinate

Missing or arbitrary information

Many facts are effectively arbitrary: an obscure person's birthday, the exact page number of a quotation, a specific API parameter. If such a fact appeared rarely or never in training data, the model has no reliable internal representation of it — yet the prompt still demands an answer, and the machinery that produces text has no "I don't know" state unless it has been trained to use one. The result is a statistically shaped guess wearing the costume of a fact. Why language models hallucinate | OpenAI

Training and evaluation reward confident guessing

One influential explanation frames hallucination as an incentive problem rather than a mysterious defect. Standard benchmarks are scored like multiple-choice exams: a correct answer earns a point, an abstention earns nothing, and a wrong answer also earns nothing. Under that scoring, guessing strictly dominates saying "I'm not sure," so models optimised against such benchmarks learn to always produce an answer. The associated research argues that hallucinations originate in the statistical pressures of pretraining and then persist because evaluation procedures reward guessing over acknowledging uncertainty, and it suggests that scoring schemes penalising confident errors more than expressions of uncertainty would shift model behaviour. Why language models hallucinate | OpenAI [2509.04664] Why Language Models Hallucinate

Data quality, prompts, and generation settings

Other contributing factors compound the core problem:

• Noisy or biased training data. Errors, outdated claims, and contradictions in the corpus get absorbed and reproduced. Web-scale data guarantees some of this.
• Knowledge cutoffs. A model asked about events after its training data ends may fabricate rather than decline, unless it has search or retrieval available.
• Ambiguous or leading prompts. A question containing a false premise ("summarise Einstein's 1961 paper on…") invites the model to elaborate on something that doesn't exist. Prompts asking for a fixed number of examples or citations pressure the model to fill quotas.
• Decoding randomness. Sampling settings that favour variety over the highest-probability continuation increase novelty and, with it, the chance of unsupported detail.
• Long contexts and long outputs. Errors accumulate across multi-step reasoning and long documents; a single early fabrication can be carried forward as an assumption.
• Sycophancy. Models tuned to be agreeable may adopt a user's incorrect assertion rather than contradict it. What Are AI Hallucinations? | IBM Survey and analysis of hallucinations in large language models - PMC

What Hallucinations Cost in Practice

Two widely reported incidents illustrate the practical stakes and the way liability tends to land on the human or organisation deploying the system.

In the US federal case Mata v. Avianca, attorneys submitted a brief containing case citations generated by ChatGPT. Several of the cited decisions did not exist; the model produced case names, reporter citations, and quotations in the correct format for authorities that had never been decided. The court sanctioned the attorneys and their firm. The episode is instructive because format-correctness is exactly what a language model is good at, and exactly what a busy reader uses as a proxy for authenticity. Mata v. Avianca, Inc.

In Canada, British Columbia's Civil Resolution Tribunal held Air Canada liable after its website chatbot described a bereavement-fare policy that did not match the airline's actual rules, leading a passenger to buy a ticket on the mistaken belief that he could claim a discount retroactively. The tribunal rejected the argument that the chatbot was a separate entity responsible for its own statements and awarded the passenger damages of about $812 (Canadian). The ruling is frequently cited for the principle that organisations remain accountable for information their AI systems give customers. BC Tribunal Confirms Companies Remain Liable for ... Airline held liable for its chatbot giving passenger bad advice

The same failure mode carries different weights in different domains: an invented statistic in a blog post is embarrassing, an invented drug interaction or dosage is dangerous, and an invented compliance requirement can be expensive. Health, legal, financial, and safety-critical questions warrant verification against primary sources and review by a qualified professional, regardless of how confident the output sounds.

Reducing Hallucinations

No known technique eliminates hallucination from current generative models, and claims of a "hallucination-free" system should be treated sceptically. What is achievable is a meaningful reduction in rate and, just as importantly, a workflow that catches the errors that remain.

Ground the model in real sources. Retrieval-augmented generation (RAG) supplies relevant documents at query time and asks the model to answer from them, which converts many open-ended factual questions into reading-comprehension tasks. Grounding narrows the failure surface but does not close it: models can still misread, over-generalise, or blend retrieved text with parametric memory, so requiring quotations and passage-level citations that a reader can check is more useful than a bare answer. What Are AI Hallucinations? | IBM

Design prompts that permit uncertainty. Explicitly allowing "I don't know," asking the model to separate what the source supports from what it is inferring, and avoiding prompts that impose arbitrary quotas all reduce pressure to invent. On the evaluation side, the argument that scoring should penalise confident errors more heavily than abstentions applies equally to internal quality metrics: if your acceptance test never rewards a hedged answer, you are training your own process to prefer guesses. Why language models hallucinate | OpenAI

Verify mechanically wherever possible. Some claims are cheap to check automatically: does the cited case, DOI, URL, or package version exist? Does the code compile and pass tests? Does the number reconcile with the spreadsheet? Building these checks into the pipeline catches a large share of fabrications without human effort.

Use cross-checking and human review proportionate to risk. Comparing independent generations, asking a second model to audit the first against sources, and routing high-stakes output to a subject-matter expert are all standard mitigations. Constraining scope — a narrow assistant over a curated corpus rather than an open-ended oracle — reliably lowers exposure.

Adjust generation and system settings. Lower-randomness decoding, tighter system instructions, and explicit refusal policies for out-of-scope topics reduce inventive drift, at some cost in creative range.

Measurement and its limits

Hallucination rates are frequently quoted, but they depend heavily on the task, the domain, the prompt style, the grounding available, and the definition of a hallucination being used. A summarisation-faithfulness score and an open-domain factuality score measure different things, and neither transfers cleanly to your own workload. Comparative figures published by vendors or leaderboards are best read as directional signals under a specific test design, not as a property of the model in general. Detection is also asymmetric: fabricated content is often easier for the model to produce than for a reader to falsify, which is precisely why the phenomenon causes harm at scale. The practical posture is to treat generative output as a draft that carries an unknown but non-zero error rate, and to build verification into the point where the output becomes a decision. Survey and analysis of hallucinations in large language models - PMC Survey of Hallucination in Natural Language Generation