Fine-tuning
Continued training of a pre-trained model on a smaller, task-specific dataset to adapt its weights for a particular domain, task, or output format.
A pre-trained LLM has broad knowledge but generic behavior. Fine-tuning updates the model's weights on a curated dataset to shift it toward a specific format, style, domain, or task without starting training from scratch.
When to fine-tune vs prompt-engineer
Fine-tuning makes sense when:
- The task requires a consistently specific output format that prompting alone struggles to maintain
- Domain-specific language or jargon needs to be baked in (medical notes, legal contracts, internal terminology)
- Latency or cost is critical - fine-tuned smaller models often outperform large models on narrow tasks
- You have hundreds or thousands of labeled examples
If you have fewer than 50 examples, start with few-shot prompting and retrieval-augmented generation first.
Fine-tuning approaches
- Full fine-tuning: Updates all weights. Most expensive but most flexible. Requires significant GPU memory.
- LoRA / QLoRA: Adds small adapter layers; only those are updated. 10-100x cheaper than full fine-tuning. The dominant approach for open-source models.
- RLHF / DPO: Trains the model to prefer outputs matching human preferences. Used by Anthropic, OpenAI, and others to improve chat behavior. DPO has gained adoption as a simpler alternative to RLHF.
Fine-tuning services and tools
Hosted fine-tuning has narrowed: OpenAI has closed fine-tuning to new users and is winding it down, Mistral has deprecated its fine-tuning API, and Anthropic offers none for current Claude models. Together AI and Tinker offer LoRA fine-tuning of open-weight models, Google Vertex AI tunes some Gemini Flash models. Anthropic does not offer general fine-tuning of its current Claude models. For open-source models, Hugging Face's PEFT library with LoRA is the standard toolkit. TRL (Transformers Reinforcement Learning) is widely used for preference tuning such as DPO, and vLLM is the common choice for serving the fine-tuned model afterwards.
Common mistakes
Overfitting on too small a dataset causes catastrophic forgetting, where the model loses general capabilities. Data quality matters far more than quantity - 200 high-quality examples beat 10,000 noisy ones. Monitor validation loss and test on out-of-distribution examples to catch degradation early.
Related terms
Models relevant to Fine-tuning
Step 5 Preview
StepFun's 600B open-weight multimodal MoE, scoring 44 on the Artificial Analysis index
View model →Muse Spark Open
Meta's open-weight release of Muse Spark: an 816B multimodal MoE
View model →Nemotron 3.5 Super VL
NVIDIA's open 120B multimodal reasoning model for agents, sized for a single GPU
View model →Kolibri-1
A German-English open-weight MoE from Aleph Alpha, pitched on European AI sovereignty
View model →