Term
SFT (Supervised Fine-Tuning)
SFT is the supervised post-training of a pretrained language model on a dataset of input-output pairs — the step that turns a base model into a usable assistant.
SFT — explained in more detail
A freshly pretrained language model (pretrained base) is a pure text completer: ask it “How does RAG work?” and it might produce the next sentence of a forum post. SFT is the training step that adapts this model to concrete question-answer or instruction-response pairs. The dataset consists of thousands to millions of examples — usually written or curated by humans. Training is classic supervised learning with cross-entropy loss on the response tokens. The result: the model learns that questions are followed by answers, that code is answered with code, and that it should reply politely, helpfully, and in a defined format. SFT is today the first post-training step of nearly every chat model, followed by a preference optimisation like RLHF, DPO or ORPO.
Example / Practical context
Practical case: an open-source base model (e.g. Llama base) should function as a customer support assistant for a specific product. Step one is SFT — a few thousand curated support tickets as question-answer pairs, trained efficiently via LoRA or QLoRA. The result is a model that hits the company’s tone and uses product-specific terminology correctly — often significantly more precise than a generic chat model with RAG bolted on.
Distinction from related concepts
Pretraining is the upstream step: the model learns language from raw text via next-token prediction, without supervised labels. RLHF, DPO and ORPO are preference methods that run after SFT — they adjust the model based on “answer A is better than B” judgments rather than fixed reference answers. Instruction tuning is a variant of SFT where the training data deliberately covers diverse instructions across many task types.
Entdecke mehr
AI Workflows by Keyword: How We Make Recurring Routines Enforceable
A typed keyword triggers a fixed AI routine — and every single step must be committed before the next one appears. Why that's the actual trick.
GlossarDPO (Direct Preference Optimization)
DPO is a fine-tuning method that aligns language models directly to human preference pairs — without a separate reward model and without reinforcement learning.
LexikonFine-Tuning Explained — When, How, With What
When fine-tuning pays off, which methods exist (SFT, DPO, LoRA, QLoRA), and what AMD vs. NVIDIA hardware actually means in practice.