A model does not do what you want. There are three broad ways to fix that, and teams routinely reach for the most expensive one first because it sounds the most serious.
They are better understood as a ladder. Start at the bottom and stop as soon as the problem goes away.
Rung one: change the prompt
Cheapest, fastest, reversible in a minute. Instructions, a clear output format, and two or three worked examples in the prompt itself.
People dismiss this as unserious, which is a mistake. A large share of “the model cannot do this” turns out to be “we asked badly”. Vague instructions, no examples, an unspecified output shape, and contradictory requirements buried in a long preamble.
Before concluding a model cannot do something, write the instruction the way you would write it for a competent new colleague who cannot ask follow up questions. That framing alone fixes a surprising amount.
Stop here when the model can already do the task and just needed telling properly.
Rung two: give it the right information
If the failure is that the model does not know something, no amount of instruction will help. It needs the facts in front of it.
That is retrieval. Look up the relevant material, put it in the prompt, ask the question against it. It handles private documents, current data, and anything that changes often.
Stop here when the problem was missing knowledge rather than missing capability. In my experience this is most of them.
Rung three: fine tune
Now you are changing the model’s weights on your own examples. It is the right tool for a narrower set of problems than its reputation suggests.
Fine tuning is good at teaching form: a house style, a rigid output structure, a domain vocabulary, a classification boundary that is hard to describe but easy to demonstrate. It is also good at making a smaller, cheaper model perform a specific task about as well as a large one, which is a real cost lever at volume.
Fine tuning is bad at teaching facts. Training on your documentation does not give the model reliable recall of it. It gives the model a strong impression of the shape of your documentation, which is exactly the ingredient for fluent, confident, wrong answers.
Use it when you need consistent form, or you need a small model to punch above its weight, and you have several hundred good examples.
The cost nobody budgets for
The training run is the small part. The real cost of fine tuning is the dataset, which needs building, cleaning and maintaining, and the version management, because you now own a model artifact that has to be evaluated, deployed, rolled back and eventually retired.
You have taken on a dependency you have to maintain. Retrieval, by contrast, updates when you update a document.
They combine, and usually should
The framing of “which one” is slightly wrong. A mature system typically uses all three: a carefully written prompt, retrieval supplying current facts, and possibly a fine tuned smaller model handling one well defined step cheaply.
The ladder is not about picking a winner. It is about not paying for rung three when rung one would have done.
The rule I keep
If the model cannot do the task at all, that is a capability problem and fine tuning might help. If it can do the task but does it inconsistently, that is a prompt problem. If it does the task well but gets the facts wrong, that is a retrieval problem.
Diagnosing which of those three you have, before choosing a tool, saves more time than any other habit I know of in this work.