Evaluation

Recall (hit rate, sensitivity) measures the share of all actually existing relevant cases that a system finds. Formula: relevant cases found divided by all relevant cases that truly exist. Requires a known ground truth.

An AI benchmark is a standardized task suite that makes language models comparable — same prompts, same scoring, transparent results for reasoning, knowledge, or coding.

Context Precision is a RAG evaluation metric (0–1) measuring how well the retriever ranks relevant contexts above irrelevant ones — the focus is ranking quality, not raw hit count.

Context Recall is a RAG evaluation metric (0–1) measuring how completely the retriever supplied the facts needed for the gold answer — every missing fact lowers the score.

Evals are systematic tests for LLM applications — fixed test cases, automatic scoring of the responses, result as a score. The foundation for objectively comparing prompt or model changes.

Faithfulness is a RAG evaluation metric (0–1) measuring how many claims in a response can actually be derived from the retrieved contexts — a direct hallucination indicator.

LLM-as-a-Judge is the practice of using a language model as an evaluator — it compares responses from other models or scores outputs against given criteria.

RAGAS is an open-source framework for automated evaluation of RAG and agent pipelines — with built-in metrics like faithfulness, context precision, and answer relevancy.