RAG vs fine-tuning: a decision cheat sheet for 2024
When to reach for retrieval, when to fine-tune, and when to do both. With concrete examples.
The default in 2024 should almost always be RAG. Retrieval is cheap, debuggable and easy to update when your knowledge changes. You can swap models without retraining anything. Fine-tuning is more powerful but it locks you into a checkpoint, a vendor, and a slow iteration loop.
Reach for fine-tuning when style, tone or format matters more than facts. Customer support agents that must speak in a specific brand voice, classifiers with stable label schemas, or code completion for an internal DSL are all good candidates. For anything fact-heavy — docs, policies, product catalogs — RAG wins.
The mature setup is both. Fine-tune the model to speak in your voice and follow your output schema; use RAG to inject the facts at query time. That separation also makes evals cleaner: you can measure style and accuracy independently.