Fine-tuning language models is no longer a laboratory ritual but has become just another piece of the product stack. What two years ago required dedicated GPUs and weeks of work is now solved with low-rank techniques, synthetic data, and managed APIs. But the landscape has changed so much that many circulating recipes remain anchored in now-obsolete practices. This review focuses on what is shifting the ground right now, without promising miracles.
What has changed in the fine-tuning cycle
The classic workflow—collecting instruction-response pairs, training all the weights, and evaluating by eye—has fragmented. Teams working with open models have adopted variants that reduce cost and risk:
- LoRA and its derivatives (QLoRA, DoRA): a minimal fraction of parameters is trained while the base model remains frozen. The official documentation of the main training libraries already presents them as the default option for fine-tuning with fewer than a million examples.
- Preference alignment: instead of imitating responses, optimization is done with preference pairs (DPO and variants). According to industry studies, this approach reduces the number of examples needed compared to traditional RLHF and avoids training a separate reward model.
- Distillation from large models: generating data with a larger model and fine-tuning a smaller one has become common practice. The important nuance: the teacher’s quality sets the student’s ceiling, and unfiltered synthetic data carries biases and hallucinations.
A realistic workflow, step by step, with the pitfalls pointed out
1. Define the target behavior before touching data
The most repeated mistake I see in teams that are starting out: launching training without a written specification of what the model should do and, above all, what it should not do. Without that contract, subsequent evaluation is open to interpretation. It is worth drafting between 10 and 20 use cases with examples of expected input and output, including edge cases (ambiguous requests, out-of-domain, manipulation attempts).
2. Curate the dataset with more rigor than enthusiasm
The usual proportion in projects that work is around a few hundred to a few thousand well-labeled examples. More volume with noise makes the result worse. Three checks that are often skipped:
- Semantic deduplication: nearly identical responses with slight variations inflate the dataset and bias the fine-tuning.
- Format consistency: if 80% of the responses use one style and 20% another, the model learns both and oscillates.
- Review of annotation biases: whoever labels leaves a trace. Two different annotators produce different distributions even if they follow the same guide.
3. Choose the base and method according to the real budget
Not everything requires a 70B model. For classification tasks, entity extraction, or responses with a fixed format, a 7-8B model fine-tuned with QLoRA is usually enough and can be served on a single GPU. For complex reasoning, fine-tuning rarely pays off compared to a larger model with good prompting; there, the cost of serving the model outweighs the benefit.
A practical detail that appears in the documentation of several platforms: if the goal is to teach new knowledge (internal policies, product catalog), fine-tuning is a mediocre tool. Retrieval-augmented generation works better for that. Fine-tuning shines when you want to change the way of responding: tone, format, refusal policy, output structure.
4. Train with a separate validation set from the very first minute
Reserving between 10% and 20% of the data for validation is not optional. Overfitting in small models is fast and silent: training loss drops while responses become rigid and repetitive. Detecting that requires looking at generation metrics, not just loss.
5. Evaluate with rubrics, not with impressions
This is where most projects founder. An honest evaluation combines:
- Automated tests of format and constraints (does it respect the JSON schema? does it cite the source? does it refuse what it should refuse?).
- Blind human evaluation on a small sample, comparing the base and fine-tuned model with the same prompt.
- A regression set that does not change between iterations, to measure drift.
According to the official documentation of several evaluation providers, automated metrics like BLEU or ROUGE correlate poorly with perceived usefulness in open-ended tasks. They are useful for format, not for quality.
The case almost no one talks about: the fine-tuning that made the product worse
A team fine-tuned an 8B model to respond to support queries with a highly controlled tone. The results on the validation set were excellent. In production, the model began rejecting legitimate questions that superficially resembled those that had been marked as out of scope. The problem was not the training: it was that the dataset included many rejection examples and very few examples of ambiguous questions that should be answered. The lesson: the distribution of the dataset defines behavior as much as the content. Fine-tuning does not teach nuances that are not represented in the data.
Trends that are redefining the landscape
Small specialized models versus monoliths
The inference cost curve pushes toward 3-8B models fine-tuned for a specific task. Several infrastructure providers already offer low-rank deployments with competitive latency. The nuance: specialization has a ceiling. A small model fine-tuned for customer service does not become a general assistant no matter how much it is trained.
Fine-tuning with filtered synthetic data
Generating data with a larger model and filtering for quality before training is now widespread practice. The known risk: models trained on unverified synthetic data tend to amplify the generator’s errors. The recommendation that appears in technical forums and library documentation is to mix real data with synthetic data and maintain a manual audit on a sample.
Evaluation as a service and traceability
Regulatory pressure in some sectors has pushed toward documenting the fine-tuning process: what data, what method, what evaluation, what known limitations. This is not bureaucracy: it is what allows deciding when to retire a fine-tuned model because its behavior has drifted.
What fine-tuning does not solve
It is worth saying plainly. Fine-tuning does not fix a data problem, does not replace good prompt engineering in simple tasks, and does not guarantee that the model stops hallucinating. Nor is it the way to keep knowledge up to date: every change in internal policy would require retraining, something that retrieval-augmented generation avoids. And in very large models, the cost of serving your own fine-tune can exceed that of using a managed API with careful prompting.
The sensible decision comes down to a prior question: do I need to change how the model responds, or do I need it to know something it does not know? The first question justifies fine-tuning. The second, almost never.