ki-konzepte Evaluation
Benchmark (AI)
An AI benchmark is a standardized task suite that makes language models comparable — same prompts, same scoring, transparent results for reasoning, knowledge, or coding.
Redaktion
Evaluation
Bewertung von LLM-Qualität — Evals, Benchmarks, Judges.
-
- ki-konzepte Evaluation
Context Precision
Context Precision is a RAG evaluation metric (0–1) measuring how well the retriever ranks relevant contexts above irrelevant ones — the focus is ranking quality, not raw hit count.
Redaktion - ki-konzepte Evaluation
Context Recall
Context Recall is a RAG evaluation metric (0–1) measuring how completely the retriever supplied the facts needed for the gold answer — every missing fact lowers the score.
Redaktion - ki-konzepte Evaluation
Evals
Evals are systematic tests for LLM applications — fixed test cases, automatic scoring of the responses, result as a score. The foundation for objectively comparing prompt or model changes.
Redaktion - ki-konzepte Evaluation
Faithfulness
Faithfulness is a RAG evaluation metric (0–1) measuring how many claims in a response can actually be derived from the retrieved contexts — a direct hallucination indicator.
Redaktion - ki-konzepte Evaluation
LLM-as-a-Judge
LLM-as-a-Judge is the practice of using a language model as an evaluator — it compares responses from other models or scores outputs against given criteria.
Redaktion - ki-konzepte Evaluation
RAGAS
RAGAS is an open-source framework for automated evaluation of RAG and agent pipelines — with built-in metrics like faithfulness, context precision, and answer relevancy.
Redaktion - ki-konzepte Evaluation
Recall
Recall (hit rate, sensitivity) measures the share of all actually existing relevant cases that a system finds. Formula: relevant cases found divided by all relevant cases that truly exist. Requires a known ground truth.
Redaktion