Term
Inference
Inference is the act of running an already-trained model to turn an input into a response — production use, not training.
Inference — explained in more detail
Where training fits a model to data, inference is the process of using the finished model: a prompt comes in, tokens are computed one after another, and the output emerges. Inference is far cheaper than training but adds up quickly in production — every request consumes compute (GPU/CPU), memory, and energy.
Example / Practical context
A local Llama 3 model on a laptop runs in inference mode: llama.cpp loads the weights into RAM, the prompt is pushed through the network, tokens are sampled. With API services like the Anthropic or OpenAI API, users pay per inference, billed by input and output tokens. Knobs such as quantization, batch size, and KV-cache determine speed and cost.
Distinction from related concepts
Inference is neither training nor fine-tuning — the weights do not change during it. It is also not a simple single-class prediction: in generative models, inference is an iterative process that builds an entire output token by token.
Entdecke mehr
AI Workflows by Keyword: How We Make Recurring Routines Enforceable
A typed keyword triggers a fixed AI routine — and every single step must be committed before the next one appears. Why that's the actual trick.
GlossarReasoning Effort
Reasoning Effort is a control parameter on modern reasoning models that sets how much internal step-by-step thinking (thinking tokens) a model spends before answering — higher levels raise quality but also latency and cost.
LexikonLLM Hallucinations — Causes and Remedies
Why LLMs confidently invent falsehoods, what types of hallucinations exist, and which remedies actually help — RAG, source enforcement, verification.