Term
Fine-Tuning
Continued training of a pretrained model on a smaller, specialized dataset to adapt style, format or domain knowledge in a targeted way.
Fine-Tuning — in more detail
Fine-tuning is the process of continuing training of an already pretrained model (foundation model) on a much smaller, curated dataset. The goal is not to teach the model brand-new general knowledge, but to sharpen behavior, tone, format or specific tasks. Common approaches range from full fine-tuning (all weights are updated) to parameter-efficient methods like LoRA and QLoRA, which train only a fraction of the parameters.
Example / practical context
A typical use case is a chat assistant expected to respond in a company-specific style with fixed answer structures. Instead of cramming those rules into every prompt, you collect 500 to 5,000 high-quality example dialogues and fine-tune the model on them. The result is shorter prompts, more consistent responses and often cheaper inference, because a smaller fine-tuned model can match a larger untuned one.
Delineation from similar terms
Fine-tuning changes the model weights, RAG only augments the prompt at runtime — frequently changing knowledge belongs in RAG, behavior and style usually in fine-tuning. Pretraining is the resource-intensive baseline run on massive datasets; fine-tuning builds on top of it and is orders of magnitude cheaper.
Entdecke mehr
AI Workflows by Keyword: How We Make Recurring Routines Enforceable
A typed keyword triggers a fixed AI routine — and every single step must be committed before the next one appears. Why that's the actual trick.
GlossarDPO (Direct Preference Optimization)
DPO is a fine-tuning method that aligns language models directly to human preference pairs — without a separate reward model and without reinforcement learning.
LexikonFine-Tuning Explained — When, How, With What
When fine-tuning pays off, which methods exist (SFT, DPO, LoRA, QLoRA), and what AMD vs. NVIDIA hardware actually means in practice.