Fine-tuning vs Prompt Engineering

These are the two main ways to adapt a model to your task. Prompt engineering changes what you send: instructions, examples, and context, adjustable per request with no training. Fine-tuning changes the model itself: you supply hundreds or thousands of input-output examples, the provider (or your own training run) updates the model’s weights, and you get a custom variant that behaves your way by default — no lengthy instructions required.

Knowing when to reach for which is a decision you’ll face on real projects, and the modern default is clear: prompt first. Prompting iterates in seconds, costs nothing beyond tokens, survives model upgrades, and — combined with few-shot examples and retrieval — covers the vast majority of use cases. Fine-tuning earns its overhead in specific situations: locking in a style or format that prompts can’t hold consistently, deep domain jargon, shrinking latency and cost by moving a long prompt’s worth of behavior into a small cheap model, or narrow classification at massive scale. One thing fine-tuning is not good for is teaching new facts — that’s retrieval’s job.

In practice, exhaust prompting first and keep an eval set of real inputs with expected outputs as you go. If quality plateaus, that same eval set becomes your training data: format it as JSONL conversation examples, run it through a fine-tuning API such as OpenAI’s, and measure the resulting model against the prompted baseline before switching.