Term
Adapter (PEFT)
An adapter is a small, freshly added network module injected into a frozen language model — only these modules are trained, while the base model itself stays unchanged.
Adapter — explained in more detail
The original adapter paper (Houlsby et al., 2019) introduced the principle that today shapes the whole PEFT family: small bottleneck MLPs are inserted between the layers of a pretrained model — a down-projection to a few dimensions, an activation, an up-projection. During training the original model weights stay frozen and only the adapter parameters are learned. The result: a few megabytes of trained weights per adapter. Multiple adapters can run on the same base model in parallel — one per use case, loadable at runtime. LoRA, IA3 and prefix tuning are modern descendants or variants of this idea.
Example / Practical context
Practical example: a Llama base model runs in the cloud, three different business units of a company — marketing, support, legal — each get their own adapter. At inference time the relevant adapter is selected and attached to the base model. Result: a single GPU cluster, three specialised models, without needing a full 70B replica in memory for each. Hugging Face’s peft library and inference servers like vLLM support exactly this multi-adapter setup natively.
Distinction from related concepts
LoRA is today the most popular adapter variant: instead of bottleneck MLPs, it injects low-rank matrices directly into the attention weights. Classic Houlsby adapters add a slight inference latency because they introduce extra layers — LoRA adapters can be merged into the base model after training with no latency overhead. PEFT is the umbrella term for all these approaches. In day-to-day usage “adapter” is now often used as a synonym for “LoRA adapter”.
Entdecke mehr
AI Workflows by Keyword: How We Make Recurring Routines Enforceable
A typed keyword triggers a fixed AI routine — and every single step must be committed before the next one appears. Why that's the actual trick.
GlossarDPO (Direct Preference Optimization)
DPO is a fine-tuning method that aligns language models directly to human preference pairs — without a separate reward model and without reinforcement learning.
LexikonFine-Tuning Explained — When, How, With What
When fine-tuning pays off, which methods exist (SFT, DPO, LoRA, QLoRA), and what AMD vs. NVIDIA hardware actually means in practice.