You ask an LLM a factual question and it answers confidently — with a detail that turns out to be false, or a citation that doesn't exist. This is hallucination, and it is the single biggest reason teams lose trust in generative AI. Here is why it happens and how to keep it from burning you.
Why It Happens
An LLM predicts the most plausible next words based on patterns in its training data. It has no built-in notion of truth — only of what sounds right. When it doesn't know something, it doesn't stop; it generates the most likely-looking answer, which can be fluent and completely wrong. Confidence in the tone tells you nothing about accuracy.
Fix 1: Ground It in Real Data (RAG)
The most effective fix is to stop asking the model to recall facts from memory and instead give it the facts to work from. Retrieval-augmented generation fetches relevant documents and puts them in the prompt:
Answer using ONLY the context below. If the answer
isn't in the context, say "I don't know."
Context:
"""
{retrieved documents}
"""
Question: {user question}
Fix 2: Give It Permission to Say "I Don't Know"
Models hallucinate partly because they are implicitly pushed to always answer. Explicitly allowing an out reduces invented answers:
If you are not confident or lack the information,
respond with "I'm not sure" rather than guessing.
Fix 3: Verify Anything That Matters
For high-stakes facts, don't trust a single generation. Ask the model to cite the source passage it used, and check it. Treat the output as a draft to verify, not an authority to quote.
Summary
Hallucination happens because an LLM generates plausible text, not verified truth — and fluency is not accuracy. Ground the model in real retrieved data so it answers from facts instead of memory, explicitly let it say "I don't know" so it stops guessing, and verify anything consequential against its cited source. You can't eliminate hallucination, but grounding and verification shrink it from a dealbreaker to a manageable risk.