Term
Faithfulness
Faithfulness is a RAG evaluation metric (0–1) measuring how many claims in a response can actually be derived from the retrieved contexts — a direct hallucination indicator.
Faithfulness — explained in detail
Faithfulness checks whether a generated answer is supported by the provided source contexts. The score sits between 0 and 1: 1 means every claim in the response can be backed by the contexts, 0 means none are. Computation has three steps: 1) claim extraction — the answer is broken into atomic statements. 2) claim verification — each statement is checked for logical derivability from the contexts. 3) score = supported claims / total claims.
Example / Practical use
Question: “When was Einstein born?”. Context: “Einstein was born on March 14, 1879 in Germany”. Answer A: “Einstein was born on March 14, 1879 in Germany” — every claim grounded → score 1.0. Answer B: “Einstein was born on March 20, 1879 in Germany” — the place is grounded, the date contradicts → score 0.5. Frameworks like RAGAS compute this either via an LLM judge or via a specialized classifier such as HHEM-2.1-Open.
Distinction from related terms
Faithfulness measures only adherence to the given context — not whether the answer is objectively true. An answer can be faithful to a flawed context and still be wrong in the real world. Related metrics: answer relevancy (does the answer match the question), context precision (how much of the context is relevant at all), and factual correctness (comparison against a gold answer). In production these are typically tracked together, because each metric exposes a different failure mode.
Entdecke mehr
AI Workflows by Keyword: How We Make Recurring Routines Enforceable
A typed keyword triggers a fixed AI routine — and every single step must be committed before the next one appears. Why that's the actual trick.
GlossarRecall
Recall (hit rate, sensitivity) measures the share of all actually existing relevant cases that a system finds. Formula: relevant cases found divided by all relevant cases that truly exist. Requires a known ground truth.
LexikonMeasuring LLM Quality — Evals, Benchmarks, Judges
How to tell whether an LLM system actually works: three evaluation layers from benchmarks to CI evals, plus pitfalls like Goodhart and judge bias.