Solutions for
Please select
Try the Web App
ki-konzepte

Fine-Tuning Explained — When, How, With What

Redaktion ·

Why fine-tuning matters — and why you probably don’t need it

Fine-tuning sounds like the natural next step the moment a pretrained language model stops fitting your use case. The reality is more sober: most problems teams describe as “the model doesn’t know enough” are solved more cleanly with retrieval (RAG) or better prompts. Fine-tuning becomes worth the effort when style, format, or domain vocabulary are so specific that context engineering hits a wall — or when the inference cost of a large model needs to be replaced by a smaller, tuned one.

This article maps the field: which methods exist, what they’re built for, what they cost in data and hardware, and where the typical pitfalls hide. After reading, you should be able to decide whether fine-tuning is the right tool for your project — and if so, which variant.

Core mechanics — what tuning actually does

A pretrained model has its weights shaped by billions of tokens. Fine-tuning means continuing to train those weights on a much smaller, curated dataset until the model exhibits the desired behavior: answers in a specific style, correction of domain terms, or the ability to follow a particular response format.

Three things change compared to pretraining: the dataset is small (typically 1,000 to 100,000 examples instead of billions of tokens), the learning rate is lower, and the training objective is narrower. The model doesn’t fundamentally unlearn anything — but it can absolutely overwrite knowledge from pretraining if the tuning dataset is off. This phenomenon is called catastrophic forgetting and is the most common cause of “after tuning, the model is suddenly worse at math.”

Two training goals, two families

Fine-tuning splits into two large groups with very different mechanics:

Volume training teaches the model new content or new formats. You feed in input-output pairs and let the model imitate the output. Classics: Supervised Fine-Tuning (SFT) with instruction datasets, or continued pretraining on raw domain text.

Preference alignment teaches the model which of two plausible answers is better. It receives “preferred vs. rejected” pairs and learns from the difference. Methods: RLHF (classical, but expensive), DPO (lighter, since no separate reward model is needed), and ORPO (combines SFT and preference in a single step).

In practice, both stages often run back-to-back: first SFT, so the model responds reasonably to the format at all; then DPO or ORPO, to sharpen tone and preferences.

When fine-tuning pays off — and when it doesn’t

Before booking a GPU: three tools compete for the same slot, and they solve different problems.

| Problem | Right tool | Why | |---|---|---| | Model doesn’t know your internal documents | RAG | Inject knowledge without touching the model | | Model doesn’t write in your house style | Prompt engineering, then SFT | Try the system prompt first, tune only at scale | | Model needs to deliver structured output reliably | Constrained decoding or SFT | Decoding tricks first, tuning only for hard schemas | | Inference cost of a large model is too high | SFT on a small model | Large model as teacher, small one as student (knowledge distillation) | | Model should internalize ratings / tonality | DPO/ORPO | Preference data is built for exactly this | | Domain vocabulary is missing entirely | Continued pretraining + SFT | First catch up on language, then on task format |

Rule of thumb: if you can achieve the behavior with five well-chosen examples in the prompt, you don’t need fine-tuning. If you need the same for 50,000 requests per day and the long prompt costs tokens every time, the step is worth it.

Efficient tuning — the PEFT family

Full fine-tuning, i.e. updating all weights of a model, is expensive: a 70B model needs several hundred GB of VRAM in FP16 training, because optimizer states and gradients must be held in addition to the model itself. No home machine and no single cloud server handles that.

The PEFT family (Parameter-Efficient Fine-Tuning, see PEFT) solves the problem by training only a fraction of the parameters. The original weights stay frozen; small, trainable modules are slotted in between.

LoRA and QLoRA

LoRA injects two small matrices per layer, whose product is added to the original weight matrix. Only the small matrices are trained — typically 0.5 to 5 percent of the total parameters. Memory consumption drops drastically, training quality is nearly identical to full tuning for most applications.

QLoRA takes it a step further: the base model is loaded in 4-bit quantization, while the LoRA adapter itself runs in 16-bit. This lets a 70B model fit on a single 48 GB GPU. The quality loss from 4-bit quantization is measurable but acceptable for most tuning tasks.

Adapters and prefix-tuning

Adapters are dedicated small network layers inserted between the existing layers. They predate LoRA and are used less often today, but live on in specialized settings (multi-task tuning, very small models).

Prefix-tuning prepends trainable “virtual tokens” to the input and doesn’t touch the model weights at all. Very lightweight, but quality usually trails LoRA — today more of a research than a production tool.

Hardware: AMD vs. NVIDIA in practice

If you want to train, you have to face the hardware reality.

NVIDIA is the path of least resistance. CUDA, PyTorch, and the entire ecosystem (bitsandbytes for quantization, Flash Attention, vLLM for inference) are at home there. An A100 or H100 is the reference platform; consumer GPUs like the RTX 4090 (24 GB) handle QLoRA up to ~30B models, an RTX 6000 Ada (48 GB) up to ~70B.

AMD is catching up via ROCm but isn’t on equal footing. PyTorch officially supports ROCm, many tuning libraries run natively — but: kernels for quantization are often NVIDIA-only or lag behind, Flash Attention arrived later on AMD with limitations, and memory optimizations like bitsandbytes-NF4 don’t work on ROCm out of the box. If you have an MI300X or W7900, you can run LoRA and QLoRA, but expect more configuration overhead.

Rule of thumb: for a first tuning project, NVIDIA is the pragmatic choice. AMD pays off when the hardware is already in place or when data sovereignty / vendor diversity is strategically important.

Tools at a glance

| Tool | Strength | What for | |---|---|---| | Hugging Face TRL | Standard library for SFT, DPO, ORPO, RLHF | Full method stack, good docs, NVIDIA + AMD | | Unsloth | 2–5× faster training, less VRAM | LoRA/QLoRA on a consumer GPU, NVIDIA-focused | | Axolotl | Config-driven, many models out of the box | Reproducible runs, team setups | | Together AI / Fireworks | Managed fine-tuning via API | When no local GPU is available | | OpenAI Fine-Tuning API | Closed-model tuning on the GPT family | When an OpenAI model has to be the base |

Hugging Face TRL is the default for in-house tuning, Unsloth the accelerator for smaller setups, and managed providers the bridge when GPU procurement is the bottleneck.

Pitfalls

Three problems show up in nearly every tuning project at least once.

Catastrophic forgetting. A long SFT run on a narrow dataset makes the model unlearn general knowledge. It only becomes visible during eval on out-of-domain tasks — math, code, general reasoning. Countermeasures: lower learning rate, fewer epochs, mix in general examples, or use PEFT instead of full fine-tuning.

Bad data quality. Tuning is merciless about learning data errors. Inconsistent answers, wrong formats, typos in “golden” examples — all of it is absorbed. More data doesn’t help if the data is wrong. Better 1,000 clean examples than 10,000 dirty ones.

Eval gaps. Many teams tune, see “sounds good” in a spot check, and ship. Three weeks later the product team reports regressions nobody saw coming. Before tuning: build an eval set that covers the use case; before shipping: measure against the eval set and compare with the base model. Without this step, tuning is a blind flight.

A worked cost example

Say you want to tune an 8B model on 10,000 domain examples, three epochs.

| Variant | VRAM needed | Time on RTX 4090 | Cloud cost (A100, 80 GB) | |---|---|---|---| | Full fine-tuning (FP16) | ~120 GB | not possible | ~6 h × $2.70 ≈ $16 | | LoRA (FP16 base) | ~28 GB | ~4 h | ~3 h × $2.70 ≈ $8 | | QLoRA (4-bit base) | ~10 GB | ~5 h | ~4 h × $2.70 ≈ $11 |

On a local RTX 4090 (24 GB), full tuning isn’t feasible, but QLoRA runs comfortably and LoRA just barely. On a cloud A100, full tuning is doable but costs roughly twice as much as LoRA with barely measurable quality gain — and the adapter is only ~50 MB instead of 16 GB for the full weights file.

At 70B the picture shifts: full tuning needs multi-GPU setups (4–8× A100), QLoRA fits on a single 48 GB card. That is precisely why QLoRA has become the default for large models.

FAQ

How much data do I need?
For SFT with a clear format, 500–2,000 examples often suffice if they're clean. For DPO, 1,000–5,000 preference pairs. Continued pretraining needs significantly more — more like 100 MB+ of raw text.
Can I use synthetic data?
Yes, this is the standard route when not enough real data is available. A larger model generates training examples that you then filter manually or with a judge model. Risk: mode collapse, when the generator carries the very weaknesses you wanted to fix.
Should I fine-tune an open-weight model or use the OpenAI API?
Open-weight is cheaper per token, you keep the weights, but you run inference yourself. OpenAI tuning is more convenient but more expensive per token, and you can't export the tuned model.
What's the difference between instruction tuning and SFT?
Instruction tuning is a special case of SFT, where the training examples are explicitly "instruction → answer" pairs. The terms overlap; in practice it's the same procedure with a different dataset style.

Conclusion

Fine-tuning is a precise tool, not a universal cure. Before you tune, check whether RAG (for knowledge) or prompt engineering (for style and format) solves the problem more cheaply. If tuning is the right path, start with LoRA or QLoRA — full fine-tuning only pays off once the adapter variant hits limits, which doesn’t happen for most use cases.

The hardware choice today is pragmatic: NVIDIA for the standard case, AMD when the cards are already in-house and a certain amount of configuration overhead is acceptable. On the tools side, Hugging Face TRL plus Unsloth is the fastest entry point, managed services the bridge when no GPU is available.

What remains is the least spectacular recommendation in this article: build the eval before you tune. Anything else is optimizing components you can’t measure at the end — which makes it not fine-tuning, but hope.

Themenuebersicht

KI-KonzepteTraining & Fine-Tuning