Term
Top-p / Top-k
Top-p (nucleus sampling) and top-k are sampling strategies for LLMs that decide which token candidates to draw from at each step — together with temperature they control the creativity of the output.
Top-p / Top-k — explained in more detail
At every generation step a language model computes a probability distribution over all possible next tokens. Top-k truncates that distribution by keeping only the k most likely candidates — typical values are between 20 and 50. Top-p (also called nucleus sampling) picks a dynamically sized set: it takes the most likely tokens whose cumulative probability reaches a threshold p (e.g. 0.9). The model then samples from this reduced set, usually weighted by the remaining probability mass.
Example / Practical context
For an unambiguous question (“Capital of France”) the probability mass concentrates on a few tokens — top-p at 0.9 almost always returns “Paris”. For an open-ended task (“Write a marketing copy”) the distribution spreads wider and top-p yields more variety. Default values across the major APIs typically sit at top_p = 1.0 or 0.95 with top_k disabled; the main control is temperature. For reproducible outputs you also set a seed.
How it differs from related terms
Temperature scales the probability distribution before sampling without limiting the candidate set — top-p and top-k cut the set itself. Greedy sampling always picks the most likely token (equivalent to top_k = 1) and is deterministic. Beam search keeps several paths in parallel and is mostly used for translation and shorter outputs today, less for open-ended LLM generation.
Entdecke mehr
AI Workflows by Keyword: How We Make Recurring Routines Enforceable
A typed keyword triggers a fixed AI routine — and every single step must be committed before the next one appears. Why that's the actual trick.
GlossarReasoning Effort
Reasoning Effort is a control parameter on modern reasoning models that sets how much internal step-by-step thinking (thinking tokens) a model spends before answering — higher levels raise quality but also latency and cost.
LexikonLLM Hallucinations — Causes and Remedies
Why LLMs confidently invent falsehoods, what types of hallucinations exist, and which remedies actually help — RAG, source enforcement, verification.