Token Context Calculator

Tokens are the basic units LLMs process. One token is roughly 4 characters or 0.75 words in English. Common words like 'the', 'is', 'a' are single tokens. Longer words like 'understanding' may be 2-3 tokens. Code and non-English text often use more tokens per word. Most API providers use BPE (Byte Pair Encoding) tokenization, with slight differences per model family.

When your input exceeds the model's context window, you get an error (context length exceeded) or the model silently truncates older parts of the conversation. The model cannot 'remember' anything outside its context window — it's stateless per API call. For long documents, use chunking, RAG, or models with large context windows (Gemini 1.5 Pro: 1M tokens, Claude: 200K tokens).

Input tokens are everything you send to the model: system prompt + conversation history + current user message. Output tokens are what the model generates in its response. APIs bill them separately, with output typically 3-5x more expensive than input. A 1000-token input generating a 500-token output costs: (1000 × input_price) + (500 × output_price).

Key strategies: (1) Compress system prompts — remove redundant instructions. (2) Summarize chat history instead of sending full logs. (3) Use structured data formats (JSON vs prose) — JSON is often more token-efficient. (4) Enable prompt caching (Anthropic, OpenAI) for repeated system prompts. (5) Use smaller models for simple tasks. (6) Truncate retrieved documents to relevant sections only.

Context utilization is the percentage of a model's maximum context window that your current input uses. At 50%, you have headroom for conversation history or long outputs. At 90%+, you risk truncation or errors. A healthy target for most applications is keeping inputs under 70% of the context window to leave room for the model's response.