Fine-Tune Data Calculator

It depends on task complexity and desired quality. Simple classification tasks can work with 50–200 high-quality examples. Complex generation tasks (summarization, code generation) typically need 500–5,000 examples for good results and 5,000–50,000 for excellent results. Quality matters far more than raw quantity — 200 excellent examples outperform 2,000 mediocre ones.

OpenAI's fine-tuning API accepts JSONL files where each line is a JSON object with a 'messages' array in chat format. Each message has a 'role' (system/user/assistant) and 'content' field. The assistant's response is what the model learns to produce. Each line = one training example.

Training tokens = sum of all tokens across all messages in all training examples. A 500-example dataset where each example has ~300 tokens would be approximately 150,000 training tokens. OpenAI charges per training token; Llama fine-tuning on your own hardware doesn't have a per-token cost.

Full fine-tuning updates all model weights — compute-intensive, requires large GPUs, but gives maximum flexibility. LoRA (Low-Rank Adaptation) adds small trainable matrices to attention layers, reducing trainable parameters by 99%+. LoRA is the standard for fine-tuning on consumer hardware and is often nearly as good as full fine-tuning for most tasks.

Try prompt engineering first — it's faster and cheaper. Fine-tune when: (1) you need consistent format/style that prompting can't reliably achieve, (2) you're hitting token limits with few-shot examples, (3) latency is critical and you can't afford large prompts, or (4) you need off-distribution behavior the base model can't generalize to.

Typically 3–5 epochs for fine-tuning. More epochs increase memorization risk. For small datasets (<500 examples), use fewer epochs (1–2) with early stopping. OpenAI's default is 3 epochs. Monitor validation loss — stop when it starts increasing (overfitting signal).