ki-konzepte Training & Fine-Tuning
Adapter (PEFT)
An adapter is a small, freshly added network module injected into a frozen language model — only these modules are trained, while the base model itself stays unchanged.
Redaktion
Training & Fine-Tuning
Verfahren zum Trainieren und Anpassen von Sprachmodellen.
-
- ki-konzepte Training & Fine-Tuning
Catastrophic Forgetting
Catastrophic forgetting describes the effect that a neural network loses previously acquired knowledge — wholly or partially — when it is trained further on new data; a key problem when fine-tuning language models.
Redaktion - ki-konzepte Training & Fine-Tuning
Continued Pretraining
Continued pretraining is the further training of an already pretrained language model on large amounts of domain text — before any classic fine-tuning begins.
Redaktion - ki-konzepte Training & Fine-Tuning
DPO (Direct Preference Optimization)
DPO is a fine-tuning method that aligns language models directly to human preference pairs — without a separate reward model and without reinforcement learning.
Redaktion - ki-konzepte Training & Fine-Tuning
Fine-Tuning
Continued training of a pretrained model on a smaller, specialized dataset to adapt style, format or domain knowledge in a targeted way.
Redaktion - ki-konzepte Training & Fine-Tuning
Instruction Tuning
Instruction tuning is a variant of supervised fine-tuning in which a language model is trained on a broad mix of instruction-response pairs — so it reliably follows instructions across arbitrary tasks.
Redaktion - ki-konzepte Training & Fine-Tuning
Knowledge Distillation
Knowledge distillation is a training technique in which a small student model learns from the behaviour of a large teacher model — aiming for comparable quality at significantly lower resource cost.
Redaktion - ki-konzepte Training & Fine-Tuning
LoRA
Low-Rank Adaptation — parameter-efficient fine-tuning that trains only small additional matrices instead of all model weights.
Redaktion - ki-konzepte Training & Fine-Tuning
ORPO (Odds Ratio Preference Optimization)
ORPO is a training method that unifies supervised fine-tuning and preference optimisation in a single step — leaner than the classic SFT-plus-RLHF pipeline.
Redaktion - ki-konzepte Training & Fine-Tuning
PEFT (Parameter-Efficient Fine-Tuning)
PEFT bundles methods that adapt a pretrained LLM by training only a small fraction of its weights — typically under 1%, instead of all billions of parameters.
Redaktion - ki-konzepte Training & Fine-Tuning
Prefix Tuning
Prefix tuning is a PEFT method that trains a sequence of learned "virtual tokens" prepended to the input rather than any model weights — the base model itself stays frozen.
Redaktion - ki-konzepte Training & Fine-Tuning
QLoRA
Extension of LoRA that quantizes the base model to 4 bits — allowing even very large LLMs to be fine-tuned on a single GPU.
Redaktion - ki-konzepte Training & Fine-Tuning
RLHF
Reinforcement Learning from Human Feedback — training procedure that aligns models with helpful and safe behavior using human preference comparisons.
Redaktion - ki-konzepte Training & Fine-Tuning
SFT (Supervised Fine-Tuning)
SFT is the supervised post-training of a pretrained language model on a dataset of input-output pairs — the step that turns a base model into a usable assistant.
Redaktion - ki-konzepte Training & Fine-Tuning
Synthetic Data
Synthetic data is artificially generated training or evaluation data — typically produced by a language model itself to extend datasets without needing real-world sources.
Redaktion