AI Testing

LLM Evaluation Metrics: How to Measure AI Model Quality

HelpMeTest

HelpMeTest

— 7 min read
LLM Evaluation Metrics: How to Measure AI Model Quality

Measuring the quality of a large language model output is fundamentally different from measuring traditional software behavior. There is no single assert response == expected — you are comparing probabilistic text against a range of acceptable answers. Getting this right requires a layered approach: automated metrics for scale, model-based judges for nuance, and human evaluation for ground truth.

This guide covers the full spectrum of LLM evaluation metrics, when to use each, and how to assemble them into a practical quality measurement system.

Why LLM Evaluation Is Hard

Traditional software testing has binary outcomes: a function either returns the correct value or it doesn't. LLMs produce free-form text where:

  • Multiple phrasings can be equally correct
  • Factual accuracy requires external knowledge to verify
  • Tone, coherence, and helpfulness are subjective
  • The same model can give different answers to identical prompts

This uncertainty is not a bug — it is the nature of generative AI. But it means your evaluation strategy must be probabilistic, multi-dimensional, and calibrated over many samples.

The Three Tiers of LLM Evaluation

Tier 1: Reference-Based Metrics (Automated)

These metrics compare model output against a known reference answer. They are fast, cheap, and deterministic — ideal for regression testing and CI/CD pipelines.

BLEU (Bilingual Evaluation Understudy)

Originally designed for machine translation, BLEU measures n-gram overlap between generated text and one or more reference translations.

from nltk.translate.bleu_score import sentence_bleu, SmoothingFunction

reference = [["the", "cat", "sat", "on", "the", "mat"]]
hypothesis = ["the", "cat", "is", "on", "the", "mat"]

score = sentence_bleu(reference, hypothesis, smoothing_function=SmoothingFunction().method1)
print(f"BLEU score: {score:.4f}")  # 0.5765

BLEU scores range from 0 to 1. Scores above 0.4 are generally considered good for translation tasks. However, BLEU penalizes paraphrasing — a semantically equivalent answer with different wording will score poorly.

ROUGE (Recall-Oriented Understudy for Gisting Evaluation)

ROUGE is better suited for summarization tasks. It focuses on recall — how much of the reference appears in the hypothesis.

  • ROUGE-1: Unigram overlap
  • ROUGE-2: Bigram overlap
  • ROUGE-L: Longest common subsequence
from rouge_score import rouge_scorer

scorer = rouge_scorer.RougeScorer(['rouge1', 'rouge2', 'rougeL'], use_stemmer=True)
scores = scorer.score(
    target="The quarterly revenue increased by 15% year over year",
    prediction="Revenue grew 15% compared to last year"
)
print(scores)
# rouge1: Score(precision=0.6, recall=0.75, fmeasure=0.667)

METEOR

METEOR addresses BLEU's paraphrasing weakness by incorporating synonym matching and stemming. It correlates better with human judgement than BLEU, especially for shorter texts.

When to use reference-based metrics:

  • Regression testing between model versions
  • A/B testing prompt changes at scale
  • Continuous integration quality gates
  • Translation and summarization tasks

Limitations: These metrics assume a single "correct" answer exists and fail when multiple valid phrasings exist.

Tier 2: Embedding-Based Metrics (Semantic Similarity)

These metrics compare meaning rather than exact wording by converting text to vector embeddings.

BERTScore

BERTScore uses BERT embeddings to compute token-level similarity between hypothesis and reference. It correlates significantly better with human judgement than BLEU or ROUGE.

from bert_score import score

references = ["The restaurant was excellent and the service was fast"]
candidates = ["The food was great and they served us quickly"]

P, R, F1 = score(candidates, references, lang="en", verbose=True)
print(f"BERTScore F1: {F1.mean():.4f}")  # ~0.89

Sentence Transformers Similarity

For a lighter-weight option, sentence transformers give you semantic similarity with less compute:

from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer('all-MiniLM-L6-v2')

expected = "The capital of France is Paris"
actual = "Paris serves as France's capital city"

emb_expected = model.encode(expected, convert_to_tensor=True)
emb_actual = model.encode(actual, convert_to_tensor=True)

similarity = util.pytorch_cos_sim(emb_expected, emb_actual)
print(f"Similarity: {similarity.item():.4f}")  # ~0.92

A similarity threshold of 0.85+ typically indicates semantically equivalent answers.

Tier 3: Model-Based Evaluation (LLM-as-Judge)

Use a powerful LLM (GPT-4, Claude) to evaluate the outputs of your model. This captures nuanced quality dimensions that automated metrics miss.

Key dimensions to evaluate:

Dimension Definition Scale
Faithfulness Does the answer stick to provided context? 0–1
Answer Relevancy Does the answer address the question? 0–1
Coherence Is the response logically consistent? 1–5
Helpfulness Would a user find this response useful? 1–5
Harmlessness Does the response avoid unsafe content? Pass/Fail

Example evaluation prompt:

import anthropic

client = anthropic.Anthropic()

def evaluate_response(question: str, context: str, answer: str) -> dict:
    prompt = f"""You are evaluating an AI assistant response.

Question: {question}
Context provided to the assistant: {context}
Assistant's answer: {answer}

Rate the answer on these dimensions (return JSON):
- faithfulness: float 0-1 (does the answer only use information from the context?)
- relevancy: float 0-1 (does the answer address the question?)
- coherence: int 1-5 (is the answer logically structured?)

Return only valid JSON."""

    response = client.messages.create(
        model="claude-opus-4-6",
        max_tokens=256,
        messages=[{"role": "user", "content": prompt}]
    )
    
    import json
    return json.loads(response.content[0].text)

result = evaluate_response(
    question="What is the refund policy?",
    context="All purchases have a 30-day money-back guarantee.",
    answer="You can get a full refund within 30 days of purchase."
)
print(result)
# {"faithfulness": 0.95, "relevancy": 1.0, "coherence": 5}

RAG-Specific Metrics

Retrieval-augmented generation (RAG) systems need additional metrics beyond standard LLM evaluation.

Context Precision

What fraction of the retrieved context is relevant to answering the question?

Context Precision = Relevant retrieved chunks / Total retrieved chunks

Context Recall

Does the retrieved context contain all information needed to answer correctly?

Answer Groundedness

Is every claim in the answer supported by the retrieved context? This is different from faithfulness — groundedness checks each atomic claim individually.

Libraries like Ragas automate all of these:

from ragas import evaluate
from ragas.metrics import faithfulness, answer_relevancy, context_precision

from datasets import Dataset

data = {
    "question": ["What is the refund policy?"],
    "contexts": [["All purchases have a 30-day money-back guarantee."]],
    "answer": ["You can get a full refund within 30 days."],
    "ground_truth": ["30-day money-back guarantee on all purchases."]
}

dataset = Dataset.from_dict(data)
result = evaluate(dataset, metrics=[faithfulness, answer_relevancy, context_precision])
print(result)

Building a Practical Evaluation Pipeline

Here is a production-ready evaluation pipeline that combines multiple metric tiers:

import json
from dataclasses import dataclass
from typing import Optional
from bert_score import score as bert_score
from rouge_score import rouge_scorer

@dataclass
class EvalResult:
    rouge_l: float
    bert_f1: float
    llm_faithfulness: Optional[float]
    llm_relevancy: Optional[float]
    overall: float
    passed: bool

class LLMEvaluator:
    def __init__(self, pass_threshold=0.75):
        self.rouge = rouge_scorer.RougeScorer(['rougeL'], use_stemmer=True)
        self.pass_threshold = pass_threshold
    
    def evaluate(
        self, 
        question: str,
        expected: str, 
        actual: str,
        context: Optional[str] = None,
        use_llm_judge: bool = True
    ) -> EvalResult:
        # Reference-based
        rouge = self.rouge.score(expected, actual)
        rouge_l = rouge['rougeL'].fmeasure
        
        # Embedding-based
        _, _, F1 = bert_score([actual], [expected], lang="en")
        bert_f1 = F1.item()
        
        # LLM judge (optional, for important evaluations)
        llm_faith = None
        llm_rel = None
        if use_llm_judge and context:
            scores = self._llm_evaluate(question, context, actual)
            llm_faith = scores.get('faithfulness')
            llm_rel = scores.get('relevancy')
        
        # Weighted overall score
        if llm_faith is not None:
            overall = (0.2 * rouge_l + 0.3 * bert_f1 + 
                      0.25 * llm_faith + 0.25 * llm_rel)
        else:
            overall = 0.3 * rouge_l + 0.7 * bert_f1
        
        return EvalResult(
            rouge_l=rouge_l,
            bert_f1=bert_f1,
            llm_faithfulness=llm_faith,
            llm_relevancy=llm_rel,
            overall=overall,
            passed=overall >= self.pass_threshold
        )
    
    def _llm_evaluate(self, question, context, answer) -> dict:
        # ... (LLM judge implementation from above)
        pass

Statistical Considerations

Single-sample evaluation is unreliable with LLMs. You need statistical rigor:

Use large enough test sets. 50 samples is the minimum for meaningful statistics. 200+ gives reliable confidence intervals.

Track distribution, not just mean. A model with mean score 0.8 but 10% catastrophic failures (score < 0.3) is worse than one with mean 0.75 and minimum 0.6.

Bootstrap confidence intervals. Never report a single number — always include ±:

import numpy as np

def bootstrap_ci(scores, n_bootstrap=1000, ci=0.95):
    means = [np.mean(np.random.choice(scores, len(scores))) 
             for _ in range(n_bootstrap)]
    lower = np.percentile(means, (1 - ci) / 2 * 100)
    upper = np.percentile(means, (1 + ci / 2) * 100)
    return np.mean(scores), lower, upper

scores = [0.82, 0.91, 0.75, 0.88, 0.79, 0.93, 0.71, 0.85]
mean, lower, upper = bootstrap_ci(scores)
print(f"Mean: {mean:.3f} (95% CI: {lower:.3f}–{upper:.3f})")

A/B test model changes. Before declaring a new model version better, run a paired significance test:

from scipy import stats

baseline_scores = [0.82, 0.79, 0.88, ...]
new_model_scores = [0.85, 0.81, 0.91, ...]

t_stat, p_value = stats.ttest_rel(new_model_scores, baseline_scores)
print(f"p-value: {p_value:.4f}")  # < 0.05 = statistically significant improvement

Human Evaluation

Automated metrics are proxies. Human evaluation is ground truth. Run human eval when:

  • Launching a new model to production
  • Evaluating subjective quality dimensions (tone, creativity, empathy)
  • Calibrating your automated metrics
  • Making high-stakes deployment decisions

Side-by-side evaluation: Show evaluators two responses (baseline and new model) without labeling which is which. Ask them to rate each or pick the better one.

Annotation guidelines matter. Without clear rubrics, inter-annotator agreement falls below 60%. Define exactly what "helpful" means with examples.

Calculate inter-annotator agreement. Cohen's Kappa > 0.6 is acceptable. Below 0.4, your task definition is ambiguous.

Setting Quality Gates

Define minimum scores before deployment:

# quality-gates.yaml
metrics:
  bert_score_f1:
    minimum: 0.82
    critical: 0.75  # below this = block deployment
  
  faithfulness:
    minimum: 0.90
    critical: 0.80
  
  answer_relevancy:
    minimum: 0.85
    critical: 0.75
  
  catastrophic_failure_rate:
    maximum: 0.02  # less than 2% of responses score below 0.4

Integrate this into CI:

def check_quality_gates(eval_results, gates_config):
    failures = []
    
    mean_bert = np.mean([r.bert_f1 for r in eval_results])
    if mean_bert < gates_config['bert_score_f1']['critical']:
        failures.append(f"BERTScore {mean_bert:.3f} below critical threshold")
    
    catastrophic = sum(1 for r in eval_results if r.overall < 0.4) / len(eval_results)
    if catastrophic > gates_config['catastrophic_failure_rate']['maximum']:
        failures.append(f"Catastrophic failure rate {catastrophic:.1%} too high")
    
    if failures:
        raise ValueError(f"Quality gates failed:\n" + "\n".join(failures))
    
    print("All quality gates passed")

Continuous Monitoring in Production

Evaluation doesn't stop at deployment. Monitor live traffic:

  • Sample and evaluate 1–5% of production responses automatically
  • Track metric drift — degradation often indicates data distribution shift or upstream changes
  • Log low-confidence responses — flag cases where your evaluator gives conflicting signals
  • User feedback signals — thumbs up/down, regeneration requests, and session abandonment are weak but real quality signals

HelpMeTest supports automated monitoring for AI applications. You can set up continuous evaluation runs that sample production traffic, run your evaluation suite, and alert when quality metrics drop below thresholds — without managing evaluation infrastructure yourself.

Summary

Metric Speed Cost Captures
BLEU/ROUGE Instant Free Lexical overlap
BERTScore Fast Low Semantic similarity
LLM-as-judge Slow Medium Nuanced quality
Human eval Very slow High Ground truth

Start with BERTScore + ROUGE-L for your regression suite. Add LLM-as-judge for faithfulness and relevancy when evaluating RAG systems. Run human evaluation quarterly or before major model changes.

A well-instrumented evaluation pipeline turns "does the AI work?" from a gut feeling into a measurable, trackable engineering property.

Read more