OpenAI Token Counter

GPT-4o uses the o200k_base tokenizer, which has a larger vocabulary (~200K tokens) and produces slightly fewer tokens for the same text compared to cl100k_base (used by GPT-4 Turbo and GPT-3.5). This tool estimates tokens using character-to-token ratios (~4.2 chars/token for o200k, ~4.0 for cl100k) plus punctuation adjustments. For exact counts, use OpenAI's tiktoken library.

cl100k_base is the BPE tokenizer used by GPT-4 Turbo, GPT-3.5 Turbo, and embedding models. It has 100,277 tokens in its vocabulary. o200k_base is the newer tokenizer used by GPT-4o and GPT-4o mini, with roughly 200K tokens. The larger vocabulary means more common subwords get their own token, resulting in slightly more efficient tokenization for many languages.

This tool uses a heuristic estimation (character count / 4 ± punctuation adjustments), not the actual tiktoken library. Real tokenization considers the full BPE merge rules, which can produce different counts for words with unusual characters, code snippets, or non-English text. Expect ±5–10% variance. For exact production counts, use tiktoken directly: pip install tiktoken.

Common strategies include: (1) shortening system prompts — replace verbose instructions with concise directives, (2) removing redundant context — only include relevant conversation history, (3) using numbered/bulleted lists instead of prose, (4) switching to GPT-4o mini for tasks that don't require full GPT-4o capability, and (5) compressing repeated content with references.

Always leave 20–30% of the context window free for the model's output. For GPT-4o (128K context), keep input tokens under 90K. For GPT-3.5 Turbo (16K context), keep input under 12K. Exceeding the context window causes an API error. If you need longer context, consider GPT-4o or switch to Claude (200K) or Gemini 1.5 Pro (1M).

No. All token estimation runs entirely in your browser with JavaScript. Your prompts are never sent to any server or third party. This tool works offline once the page is loaded.