A hallucination in language model terms is when the model generates false information with confidence. It's not making up something new in the creative sense. It's generating text that is statistically probable given its training, but factually false.
The reason hallucinations are inevitable is structural: the model was trained to predict the next token based on statistical patterns in text. It was never trained to say "I don't know" or "I cannot verify this." Every example in its training data showed a human (author, journalist, conversationalist) providing an answer—true, false, partially true, outdated—but always formatted as an answer. The model learned to generate answer-shaped text, regardless of whether the answer is correct.
This is not a sign of stupidity or poor training. It's a consequence of the training objective: predicting the next token in a sequence. That objective incentivizes the model to always produce something, never to abstain.
Factual Hallucination: Generation of False Facts
The model generates claims about the world that are verifiably false.
Example: Asked "When was the Titanic discovered?", the model responds "The Titanic was discovered on April 15, 1912" (the date it sank, not when it was discovered—discovered 1985).
Or: "Who won the 2024 US Presidential Election?" generates a specific name that may be wrong, partly because the model's knowledge cutoff predates the election.
Factual hallucinations are most dangerous when they're plausible-sounding. "The capital of New Zealand is Auckland" (wrong; it's Wellington) is more dangerous than "The capital of New Zealand is the Eiffel Tower" because the first requires fact-checking expertise.
Citation Hallucination: Generating Fake Sources
The model generates citations or references that don't exist.
Example: "Provide sources for your claim about photosynthesis efficiency." Model: "According to Smith (2019) in Journal of Botanical Research, photosynthesis is X% efficient."
But no such paper exists. The model generated a fake citation because it learned the pattern: (claim → cite source), and it executed that pattern without verifying the source exists. This is especially dangerous because citations are typically treated as proof of credibility.
Internal Consistency Hallucination: Contradiction Within a Single Output
The model generates statements that contradict each other within the same response.
Example: "Alice has 5 apples. She gives 3 to Bob. How many does Alice have? Alice has 2 apples remaining. Bob now has 8 apples."
The model got the first part right (2 apples) but then added that Bob has 8 apples, which is inconsistent with the 3-apple transfer from a 5-apple starting point. The model didn't "check its work"—it just continued generating statistically probable next text.
Reasoning Hallucination: Confident Incorrect Logic
The model generates a reasoning chain that sounds logical but is actually invalid.
Example: "All birds can fly. Penguins are birds. Therefore, penguins can fly."
The model produced valid-looking syllogistic structure but applied it to a false premise (not all birds can fly). It pattern-matched the form of logical reasoning without verifying the premises.
Larger models have more parameters and better learned patterns, which might suggest they hallucinate less. And they do, slightly—but the improvement is small relative to the scaling improvement in other tasks.
Why?
1. Scale improves pattern memorization, not truthfulness verification
A larger model is better at reproducing patterns it learned from training data. If a false claim appeared frequently in training data, a larger model will reproduce it more accurately and confidently. Scale doesn't make the model better at checking whether something is true—it only makes it better at predicting what humans wrote, which includes both true and false claims.
2. Hallucination is not always distinguishable from synthesis
Sometimes what looks like hallucination is actually reasonable synthesis. If asked "What would Shakespeare write about social media?", the model generates plausible-sounding Shakespearean language about modern technology. Is this hallucination? It's not a real Shakespeare quote, but it's not claiming to be. It's synthesis. The line between synthesis and hallucination is: does the model claim its output is factual?
3. Larger models are more confident
A subtle finding: larger models produce more confident-sounding language. They use fewer hedges ("might," "could," "it's possible") and more definitive claims. This isn't because they're more accurate—it's because they learned confident-sounding language more thoroughly. So scaling can actually make hallucinations harder to detect, even if the rate is slightly lower.
The most dangerous property of language models: confidence and accuracy are decoupled.
A well-trained model can generate a completely false statement with the same confidence (mathematically, the same probability score) as a true statement. There is no built-in mechanism that would make false outputs generate lower confidence.
This is because confidence comes from the probability distribution the model learned: if two completions are equally statistically probable given the training data, the model treats them as equally confident. If both true and false claims about a topic appeared in training data, the model learned to generate both fluently and confidently.
Detecting Hallucination is Not Possible from the Model's Perspective
You cannot ask a language model "Are you confident about this?" or "Is this true?" and expect reliable answers. The model doesn't have access to a confidence signal orthogonal to its training. Asking "Is this claim true?" gets incorporated into the prompt, and the model generates more text based on its patterns. It might say "Yes, I'm confident" even when generating hallucinations, because it learned that confident-sounding language follows factual claims.
1. Fact-Check High-Stakes Claims
Any output that would affect decisions should be verified against authoritative sources. This is non-negotiable for:
2. Require Citations and Source Material
Ask the model to cite sources or reference provided context. This doesn't prevent hallucination of sources, but it creates an opportunity to verify. Better: provide the source material directly in the prompt, then ask the model to quote or reference specific passages.
Good: "Based on this article [article text], what are the main arguments?" Bad: "What are the arguments in Smith's article?" (The model might hallucinate the article.)
3. Decompose Complex Claims
For multi-part claims, ask about each part separately. This prevents the model from confidently chaining together partially-false reasoning.
Instead of: "Explain how photosynthesis works and why it's important for agriculture." Better:
Separately-asked questions give you opportunity to fact-check each piece independently.
4. Use Explicit Uncertainty Tagging
Ask the model to tag claims by confidence level: (certain, likely, plausible, speculative). This doesn't make the claims more accurate, but it creates explicit flags for outputs that require verification.
Prompt: "Explain X. For each major claim, tag it: [CERTAIN] if you're trained on verified information, [LIKELY] if based on common patterns, [PLAUSIBLE] if reasonable but unverified, [SPECULATIVE] if uncertain."
5. Consistency Checking
Ask the model the same question twice (or in different formats) and compare outputs. If two outputs contradict each other, both should be treated as uncertain.
6. Ground in Provided Context
Hallucination is reduced (though not eliminated) when the model is grounded in provided context. Instead of asking open-ended questions, provide text and ask the model to analyze it.
Prone to hallucination: "How did the 2008 financial crisis affect long-term employment?" Better: "Here's the unemployment data from 2008-2012. What patterns do you observe?" (Model works from provided data, not from training-data patterns.)
These strategies reduce hallucination harm but cannot prevent hallucination:
The fundamental limitation: the model has no ground truth. It cannot access real-time information, cannot reason about causality in ways independent of its training data, and cannot "know" what it doesn't know.
In some contexts, hallucination is indistinguishable from synthesis, and synthesis is valuable:
The risk is conflating these contexts. A brainstorm output should never be treated as research. A creative synthesis should never be treated as current information. The user's job is to know which context the output is serving.
Psychology: Overconfidence bias parallels hallucination. Humans confidently assert false beliefs without access to ground truth, similar to how models confidently generate false claims. The parallel suggests that confidence and accuracy are orthogonal in information-processing systems generally—both biological and artificial. This has implications for how we should evaluate expertise: confidence is not a signal of accuracy.
Creative Practice: Hallucination as inspiration. Writers and artists regularly generate ideas that are partially false or impossible, then use them as prompts for creative thinking. A hallucinated "historical fact" might become the seed for fiction. The distinction is: does the output claim to be factual? In fiction, plausibility without truth is valuable.
History: Historians deal with hallucination at a meta level—sources hallucinate (witnesses misremember, documents are miscopied), and historians must develop practices for detecting and accounting for it. Language models are doing the same thing at text-generation time: producing plausible-looking claims without verification. Historical methodology offers patterns for dealing with this (cross-reference, internal consistency checking, source hierarchy).
The Sharpest Implication
Hallucination is not fixable by better training of the same architecture. It's structural to the next-token-prediction objective and the fact that models have no access to ground truth. This means:
Generative Questions