Whenever a team wants an LLM to work with their proprietary data, the first instinct is almost always "let's fine-tune it." I think that instinct is usually wrong. Fine-tuning is a powerful tool aimed at a different problem than the one most people actually have, and reaching for it first wastes time and money.
Fine-Tuning Teaches Behavior, Not Facts
The critical misunderstanding: fine-tuning is good at teaching a model a style, format, or task pattern — not at teaching it new facts. Fine-tune on your support tickets and the model learns to sound like your support team, but it won't reliably recall the specific policy in ticket #4821. Worse, the facts you bake in go stale the moment your data changes, and updating means retraining.
Retrieval Solves the Actual Problem
Most teams don't want the model to sound different — they want it to know their data. That is retrieval, not fine-tuning. Put the relevant facts in the prompt at query time and the model uses them directly, always current:
Context: {retrieved company documents}
Question: {user question}
Answer using only the context above.
Change a document, and the next answer reflects it — no retraining, no cost, no delay.
The Cost and Maintenance Reality
Fine-tuning means a training pipeline, a labeled dataset, evaluation, and a re-run every time your data or a base model changes. Retrieval means an index you update continuously. One is a recurring project; the other is a running system:
# Retrieval: update one document, done vector_store.upsert(doc_id="policy-4821", text=new_policy)
Where I Land
Start with prompting. If the model needs your knowledge, add retrieval. Only reach for fine-tuning when you have a genuine behavioral need — a consistent output format, a specialized task, a tone you can't get with instructions — and you've confirmed retrieval can't deliver it. Fine-tuning is the right tool for a narrow set of problems; "make the model know our data" is almost never one of them.