RAG Chunking Calculator

Chunking splits documents into smaller pieces that get embedded and stored in a vector database. When a user queries the system, only the most relevant chunks are retrieved and added to the LLM context. The chunk size determines how much text is retrieved per result, affecting both retrieval precision and context quality.

Smaller chunks (128-256 tokens) provide precise retrieval and reduce noise, but may miss surrounding context needed to answer questions. Larger chunks (512-1024 tokens) provide more context per retrieval but may include irrelevant text. Most production systems use 256-512 tokens with 10-20% overlap.

Overlap duplicates a percentage of text between adjacent chunks to ensure concepts that span chunk boundaries aren't lost. With 20% overlap on 512-token chunks, consecutive chunks share ~100 tokens. This improves retrieval coherence but increases storage and embedding costs.

Typical values are 3-10 chunks. More chunks provide more context but consume more tokens in the LLM prompt, increasing cost. Start with 5, then tune based on your context window budget and retrieval quality. For 512-token chunks, 5 chunks = ~2,560 tokens, which is manageable in most models.

Fixed-size chunking splits at exact token counts — simple, predictable, and fast. Semantic chunking uses sentence/paragraph boundaries for more natural splits. For code, split by functions/classes. For prose, split by paragraphs. For structured data (CSV, JSON), split by rows/objects. Semantic chunking usually gives better retrieval but is slower.