LLM Fine-Tuning Cost Estimator

As of 2025, GPT-4o fine-tuning costs approximately $25 per 1M training tokens. A typical fine-tuning run with 100K examples at 500 tokens each = 50M tokens = ~$1,250 for training. Plus inference costs are higher for fine-tuned models ($3.75/1M input vs $2.50 for base). Factor in multiple training runs for experimentation.

Fine-tune when: (1) prompt engineering isn't achieving consistent style/format, (2) you need to reduce prompt length significantly (cost savings), (3) you need very specific domain knowledge baked in, (4) inference latency from long prompts is too high. Don't fine-tune for knowledge injection — RAG is better for facts that change. Fine-tune for style, format, and tone.

Full fine-tuning updates all model weights — expensive but achieves the best results. LoRA (Low-Rank Adaptation) freezes most weights and trains small adapter matrices — 10-100x less memory and compute. QLoRA adds 4-bit quantization for even lower memory. For self-hosted models, LoRA on Llama 3 70B can run on a single A100. For most use cases, LoRA results are nearly as good as full fine-tuning.

Minimum viable: 50-100 examples (for simple style changes). Good results: 500-1000 examples. Strong performance: 1000-5000 examples. Diminishing returns after 10K for most tasks. Quality beats quantity — 200 excellent examples often outperforms 2000 mediocre ones. For GPT-3.5 style tasks, 500-1000 examples typically gives solid results.

Llama 3 8B full fine-tune: 2x A100 80GB recommended. With LoRA: 1x A6000 48GB or A100 40GB. Llama 3 70B full fine-tune: 8x A100 80GB. With LoRA: 2-4x A100 80GB. Cloud costs: A100 80GB on Lambda Labs ~$1.99/hr, on AWS p4d.24xlarge (8x A100) ~$32/hr. Most fine-tuning runs take 1-24 hours depending on dataset size.