blog.
What Fine-Tuning Smaller Models Taught Me About Reasoning
- Fine-tuning
- Reasoning
- PEFT
- Evaluation
Working with smaller models removed a lot of comforting illusions from my training loop. When the parameter budget is limited, weak data curation, noisy labels, and vague evaluation show up immediately in the outputs.
That constraint turned out to be useful. It forced me to think more carefully about what I was teaching the model, how I was measuring progress, and which failures were actually about reasoning versus simple formatting or retrieval errors.
Data quality changes more than another training run
The strongest lift usually came from cleaning datasets, not from increasing run count. For reasoning tasks, I care less about the total number of examples and more about whether the examples demonstrate clear intermediate steps, tool use boundaries, and consistent completion styles. Smaller models amplify dataset inconsistencies instead of smoothing them out.
I now spend more time removing ambiguous samples, deduplicating near-copies, and checking whether the target behavior is actually learnable from the data format. That sounds less exciting than scaling a run, but it usually improves output quality faster and with far less cost.
Evaluation needs to separate reasoning from presentation
A model can fail a benchmark for the wrong reason. I have seen runs that produced strong intermediate reasoning but lost points because the answer format drifted, or because tool invocation syntax was inconsistent. If evaluation mixes reasoning quality with surface formatting, it becomes difficult to know what to fix next.
My preferred approach is layered evaluation. First I check final-task accuracy. Then I inspect reasoning traces, failure clusters, and prompt sensitivity. That decomposition gives much better direction for the next iteration than a single aggregate score ever will.
Smaller models reward tighter experimental discipline
Because smaller models train faster, they encourage shorter and more disciplined feedback loops. That means clearer experiment naming, stable validation slices, and reproducible training configs. When those pieces are missing, teams end up remembering runs by intuition instead of evidence.
The broader lesson is that reasoning performance is rarely improved by one dramatic trick. It usually comes from a steady pipeline of cleaner data, narrower objectives, and better measurement. Smaller models simply make that truth harder to ignore.