Hallucination

LLMs don't retrieve facts from a database - they generate text based on learned statistical patterns. Sometimes those patterns produce false statements with the same confident fluency as true ones. The model doesn't "know" it's wrong; it has no mechanism to verify truth, only token prediction probabilities.

Types of hallucination

• Factual hallucination: Inventing specific facts (dates, statistics, names) that don't exist or misrepresenting known facts.
• Citation fabrication: Generating plausible-sounding paper titles, DOIs, URLs, or author attributions that don't exist. This remains a persistent problem even in recent deployments.
• Instruction hallucination: Claiming to have performed an action (reading a file, executing code, accessing the web) that wasn't actually done.
• Temporal confusion: Stating outdated information as current fact, or treating training data cutoff dates as present-day knowledge.

Why models hallucinate

Training optimizes for token prediction accuracy across diverse text corpora. When asked about rare, ambiguous, or absent-from-training-data topics, models generate statistically plausible text rather than refusing to answer. The model has no internal signal to distinguish "I'm predicting based on strong evidence" from "I'm pattern-matching without grounding."

Mitigating hallucinations

• Retrieval-Augmented Generation (RAG): Ground responses in retrieved documents the model can cite, reducing fabrication of unseen facts.
• Tool use: Allow the model to call search APIs, databases, or code execution rather than generate facts from weights.
• Constrained output: Request JSON with required fields or structured formats; constraints reduce but don't eliminate hallucinations.
• Self-consistency: Generate multiple completions and aggregate answers; more useful for reasoning tasks than factual recall.
• Explicit uncertainty: Prompt the model to express confidence levels and refuse claims below a threshold, though model confidence doesn't reliably correlate with accuracy.