Architecture · 8 min read · Aug 12, 2026
Three ways to give a model your data.
Almost every AI project arrives at the same fork: the model is good but it does not know your business. There are three ways to fix that, they are routinely confused with each other, and choosing wrong is expensive in a way that only becomes obvious months later.
They solve different problems, not the same problem at different prices
The usual framing is that retrieval, fine-tuning and long context are three price points on one shelf. They are not. They change different things, and the reason projects pick wrong is that the question is asked as "which is best?" rather than "what is actually missing?"
| Approach | What it changes | What it cannot fix |
|---|---|---|
| Retrieval (RAG) | What the model knows right now | How it writes or reasons |
| Fine-tuning | How the model behaves, formats and decides | Facts that change after training |
| Long context | How much it can hold in one pass | Cost and latency at scale |
Say the missing piece out loud and the choice usually makes itself. It doesn't know our products is a knowledge gap — retrieval. It knows the products but writes nothing like us is a behaviour gap — fine-tuning. It handles one contract fine but not forty at once is a capacity gap — context.
Retrieval is the default, and it should be
For the overwhelming majority of business systems, retrieval is the right first answer. Your data stays in your store, it can be updated the moment it changes, access control still applies at query time, and you can show a user where an answer came from.
That last property does more work than people expect. A system that answers with a citation can be checked; a system that answers from absorbed training data cannot. In any setting where being wrong has a consequence, checkability is not a nice-to-have — it is what makes the output usable at all.
If you cannot show where an answer came from, you have not built a system anyone can be accountable for. You have built something that sounds right.
Retrieval also fails in a specific, recognisable way, and it is worth knowing the shape of it. It fails at retrieval, not at generation. When a RAG system gives a bad answer the usual cause is that the right passage was never fetched — the chunk boundaries split it, the query embedded poorly, or the document was never indexed. Teams debug the prompt for a week when the problem is one layer down.
Fine-tuning is for behaviour, and it is a commitment
Fine-tuning changes how a model responds — tone, structure, house format, consistent classification against your own taxonomy. It is genuinely good at that and genuinely bad at holding facts, because anything baked into weights is frozen at the moment you trained.
The cost people underestimate is not the training run. It is that you now maintain a model. Every base model upgrade poses a question: re-tune, or stay behind? Your evaluation set has to survive that decision, and the training data has to be reproducible months later by someone who did not build it.
Where it earns its keep is narrow, high-volume, repetitive judgement — classifying tickets into your own categories, extracting the same twelve fields from a document type you see thousands of times, enforcing an output shape that matters downstream. If you are fine-tuning to teach the model facts, you have chosen the wrong tool.
Long context is a capability, not an architecture
Large context windows removed a whole class of engineering problems, and it is tempting to treat them as a replacement for retrieval. For one-off analysis over a bounded set of documents, they often are — reading forty contracts in one pass is a real capability that did not exist before.
As a production pattern it has three costs that only show up under load. You pay for every token on every call, so stuffing a large corpus into each request multiplies your bill by your traffic. Latency scales with what you send. And attention across a very long input is not uniform — material in the middle gets weighted less reliably than material at either end, which is a quiet accuracy problem rather than a loud one.
The practical pattern is both: retrieve to narrow the field, then use a generous context to reason over what came back. That gets the freshness and auditability of retrieval with enough room to see the whole picture.
How to actually decide
Four questions, in order. Most projects stop at the first or second.
- Does the answer change? If your data updates weekly, fine-tuning is already wrong. Retrieve.
- Does the user need to check it? If yes, you need citations, which means retrieval.
- Is the gap knowledge or behaviour? Knowledge is retrieval. Behaviour, at volume, may justify fine-tuning.
- Does the task need everything at once? Cross-document reasoning over a bounded set is a genuine long-context case.
One more thing worth saying plainly: start with prompting and retrieval, measure, and only then consider fine-tuning. A surprising number of behaviour problems that get diagnosed as needing a tuned model turn out to be an instruction problem or a missing example. Tuning early buys you a maintenance burden to solve something a better prompt would have handled.
Dealing with this in your own group?
We answer scoping questions before there's a contract in sight — including the ones about cost and data handling.