The question arrives every week: do we need fine-tuning, RAG, or sharper prompt engineering? Answer with a decision tree, because cost climbs in steps: prompting runs in hours, RAG runs in tens of thousands, fine-tuning runs in retraining cycles.

Start with prompt engineering

Prompting solves most cases at near-zero cost: hours of work and iteration loops measured in minutes. Exhaust the path before climbing. Well-chosen few-shots, explicit output structure, and a system prompt versioned in the repo deliver more than most courses promise.

RAG fixes knowledge problems

Symptoms: the model lacks your internal data, the base changes every week, users demand citations with sources. That trio calls for retrieval. Initial builds run $10k to $50k (ingestion pipeline, chunking, embedding, hybrid search), and operations consume $2k to $10k monthly depending on volume and reindexing frequency.

Fine-tuning fixes behavior problems

Mirror symptoms: rigid output formats, brand tone, domain vocabulary, lower latency through distillation into compact models. Training adjusts how the model responds. Facts are another story: trained-in knowledge produces confident hallucination, where the model repeats the pattern and misses the content. Knowledge lives in retrieval; behavior lives in weights.

The cost ladder

  • Prompt engineering: $0-5k, iteration in hours, zero infrastructure
  • RAG: $10-50k build plus $2-10k/month for pipeline and search operations
  • Fine-tuning: $15-100k across curated data, training, and evals, with retraining whenever behavior drifts

The decision tree

  1. Private or current data? Go RAG.
  2. Rigid format or style? Fine-tune.
  3. Both? RAG first, distill later.
  4. Neither? Stay on prompting and invest in evals.

The classic failure mode

A team injects knowledge through fine-tuning to skip the retrieval pipeline. The model starts hallucinating with doubled confidence, and nobody can trace where each claim came from. Citations require retrieval; training offers no shortcut. When trying to save $20k of pipeline costs the credibility of the whole product, the math stops working.