AI Hallucination Testing: How to Detect and Measure LLM Fabrications
Hallucination is when an LLM presents false information as true. The model doesn't know it's lying — it's pattern-matching to produce plausible-sounding text, and sometimes plausible diverges from factual.
For most applications, hallucinations aren't a curiosity — they're a production defect. A customer support bot that invents product features, a legal assistant that cites non-existent case law, or a medical summary that fabricates dosage information can cause real harm. Testing for hallucinations before shipping is not optional.
Types of Hallucinations
Understanding what you're testing for helps you design the right tests.
Factual hallucinations — the model asserts something false as true.
"The Eiffel Tower was built in 1850." (It was completed in 1889.)
Context hallucinations — the model invents details not present in the provided context.
User provides: "Our product has 3 color options." Model outputs: "Our product comes in 5 colors including..."
Citation hallucinations — the model fabricates sources, authors, or quotes.
"According to a 2023 MIT study published in Nature..." (study doesn't exist)
Self-contradiction hallucinations — the model contradicts itself within a single response.
"The deadline is Friday." ... "Make sure to submit by Thursday."
Each type requires different detection methods.
Testing for Context Hallucinations (Most Common)
When your LLM receives context (documents, retrieved chunks, user data), it should only answer using that context. Context hallucinations happen when the model goes beyond provided information.
The standard test: provide a controlled context and verify that every claim in the output can be traced to that context.
def test_no_hallucination_beyond_context():
context = """
Product: BasicPlan
Price: $29/month
Features: 5 users, 10GB storage, email support
"""
response = product_assistant(
context=context,
question="Tell me about the BasicPlan"
)
# These facts ARE in context — model should include them
assert "29" in response or "$29" in response
assert "5 users" in response.lower() or "five users" in response.lower()
# These facts are NOT in context — model should NOT invent them
hallucinated_features = [
"API access",
"24/7 support",
"unlimited",
"priority",
"free trial",
"annual discount"
]
for feature in hallucinated_features:
assert feature.lower() not in response.lower(), \
f"Model hallucinated '{feature}' not present in context"For more sophisticated detection, use an LLM to check faithfulness:
from openai import OpenAI
client = OpenAI()
def check_faithfulness(context: str, response: str) -> dict:
"""
Returns whether each claim in the response is supported by context.
"""
result = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{
"role": "user",
"content": f"""Context: {context}
Response to evaluate: {response}
For each factual claim in the response, determine if it is:
- SUPPORTED: directly stated in context
- UNSUPPORTED: not in context (potential hallucination)
- CONTRADICTED: conflicts with context
Format your answer as JSON:
{{
"claims": [
{{"claim": "...", "status": "SUPPORTED|UNSUPPORTED|CONTRADICTED", "evidence": "..."}}
],
"faithfulness_score": 0.0-1.0
}}"""
}],
response_format={"type": "json_object"}
)
return json.loads(result.choices[0].message.content)
def test_high_faithfulness():
context = load_product_docs()
response = generate_product_summary(context)
faithfulness = check_faithfulness(context, response)
assert faithfulness["faithfulness_score"] >= 0.9, \
f"Low faithfulness: {faithfulness['faithfulness_score']}\n" + \
"\n".join([f"- {c['claim']}: {c['status']}"
for c in faithfulness["claims"] if c["status"] != "SUPPORTED"])Testing for Factual Hallucinations
For grounded facts (dates, statistics, proper names), you can build a verification test using known ground truth.
FACTUAL_GROUND_TRUTH = [
{
"question": "What year was Python first released?",
"correct_answer": "1991",
"wrong_answers": ["1989", "1995", "1993", "1994"]
},
{
"question": "What does HTTP stand for?",
"correct_answer": "Hypertext Transfer Protocol",
"wrong_answers": ["Hypertext Transfer Process", "High Transfer Protocol"]
}
]
def test_factual_accuracy():
failures = []
for case in FACTUAL_GROUND_TRUTH:
response = llm.complete(case["question"]).lower()
if case["correct_answer"].lower() not in response:
failures.append(f"Missing correct answer '{case['correct_answer']}' for: {case['question']}")
for wrong in case["wrong_answers"]:
if wrong.lower() in response:
failures.append(f"Hallucinated wrong answer '{wrong}' for: {case['question']}")
assert not failures, "\n".join(failures)This approach scales: maintain a CSV of ground-truth Q&A pairs for your domain and run it on every model update.
Testing for Citation Hallucinations
If your application generates citations or references, verify them.
import requests
from urllib.parse import urlparse
def verify_url_exists(url: str) -> bool:
try:
response = requests.head(url, timeout=5, allow_redirects=True)
return response.status_code < 400
except Exception:
return False
def test_no_hallucinated_urls():
"""Verify that any URLs in the response actually exist."""
response = research_assistant("What are good resources for learning Rust?")
url_pattern = r'https?://[^\s\)\"\'<>]+'
urls = re.findall(url_pattern, response)
dead_urls = [url for url in urls if not verify_url_exists(url)]
assert not dead_urls, f"Model generated non-existent URLs: {dead_urls}"For academic citations, you can check against known databases:
def test_no_hallucinated_doi():
"""Check that DOIs in citations are real."""
response = academic_assistant("Summarize research on transformer architectures")
doi_pattern = r'10\.\d{4,}/[^\s]+'
dois = re.findall(doi_pattern, response)
for doi in dois:
check_url = f"https://doi.org/{doi}"
assert verify_url_exists(check_url), \
f"Model hallucinated non-existent DOI: {doi}"Self-Contradiction Detection
Long LLM outputs sometimes contradict themselves. This is particularly common in document summaries and complex reasoning.
def test_no_self_contradiction():
response = llm.complete(
"Compare the advantages and disadvantages of microservices vs monolithic architecture."
)
# Use LLM to check for contradictions
contradiction_check = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{
"role": "user",
"content": f"""Read this response and identify any self-contradictions
(places where the text says two incompatible things).
Response: {response}
If you find contradictions, list them. If none, say "NO CONTRADICTIONS".
Be strict — only flag genuine logical contradictions, not nuanced trade-offs."""
}]
)
result = contradiction_check.choices[0].message.content
assert "NO CONTRADICTIONS" in result, \
f"Self-contradictions detected:\n{result}"Production Hallucination Monitoring
Tests catch hallucinations before deployment. Monitoring catches them after.
The pattern: log a sample of LLM inputs and outputs to a queue, run async hallucination checks, alert when the rate exceeds a threshold.
import asyncio
from dataclasses import dataclass
from datetime import datetime
@dataclass
class HallucinationEvent:
timestamp: datetime
input: str
output: str
context: str | None
faithfulness_score: float
suspicious_claims: list[str]
async def monitor_production_sample(llm_call_log: list[dict]) -> list[HallucinationEvent]:
"""
Async hallucination check for production logs.
Run this on a 5% sample of production calls.
"""
events = []
for call in llm_call_log:
if call.get("context"):
result = check_faithfulness(call["context"], call["output"])
if result["faithfulness_score"] < 0.8:
suspicious = [
c["claim"] for c in result["claims"]
if c["status"] == "UNSUPPORTED"
]
events.append(HallucinationEvent(
timestamp=datetime.now(),
input=call["input"],
output=call["output"],
context=call["context"],
faithfulness_score=result["faithfulness_score"],
suspicious_claims=suspicious
))
return eventsSet an alert when the hallucination rate (% of calls with faithfulness < 0.8) exceeds 5%. A sudden spike usually means a prompt change went wrong or the retrieval system is returning bad context.
Mitigation Patterns That Make Testing Easier
Some architectural choices reduce hallucinations and make them easier to test:
Constrained output with citations. Require the model to cite the specific passage it's drawing from:
SYSTEM_PROMPT = """Answer questions using ONLY the provided context.
For each claim, add a citation: [Source: exact quote from context].
If you cannot answer from the context, say "I don't have information about this."
"""Now your test can verify that every sentence has a valid citation that appears verbatim in the context.
Confidence gates. If the model's answer doesn't meet a faithfulness threshold, return a fallback:
def safe_answer(context: str, question: str) -> str:
response = llm.complete(context=context, question=question)
faithfulness = check_faithfulness(context, response)
if faithfulness["faithfulness_score"] < 0.75:
return "I'm not confident in my answer for this question. Please check the source documentation."
return responseStructured extraction over free-form generation. Instead of "summarize this contract," ask for structured extraction: {"parties": [], "effective_date": "", "key_obligations": []}. Structured outputs are easier to validate and harder to hallucinate into.
The Minimum Viable Hallucination Test Suite
If you're starting from zero, build this in order:
- Context faithfulness test — 20 cases where you know the ground truth context. Verify no facts are invented.
- Forbidden content test — list things the model should never say (competitor names, features you don't have, etc.). Verify they don't appear.
- URL/citation verification — if your app generates links or sources, verify they exist.
- Adversarial prompts — test inputs designed to make the model speculate or make things up.
That's 40-60 tests. Run them on every deployment. A hallucination caught in testing is a PR fix. A hallucination caught by a user is a support ticket, a refund, and potentially a lawsuit.
