LLM Context Window Comparison

A context window is the maximum amount of text a language model can read and consider at once — including your system prompt, conversation history, and the model's own response. It's measured in tokens. 1 token ≈ 0.75 words in English. Running out of context causes the model to lose earlier parts of the conversation.

128K tokens ≈ 96,000 words ≈ a 350-page novel. At 200K tokens (Claude, o1), you can fit roughly 150,000 words. At 1M tokens (Gemini 1.5), you can process approximately 750,000 words — the equivalent of 5–6 full-length novels or an entire codebase.

Not necessarily. Models with large context windows can technically process more text, but their ability to reason about information in the middle of very long contexts varies. This is known as the 'lost in the middle' problem. Gemini 1.5 and Claude 3.5 have shown strong performance across their full context windows in benchmarks.

Context window is how much the model can read at inference time. Training data size is how much text the model learned from during training. They are independent — a model can have a small context window but be trained on enormous datasets.

Common use cases for large context: processing entire codebases, book-length documents, long contract analysis, multi-hour transcripts, video frame analysis, or maintaining very long chat histories. 1M token context (Gemini 1.5) can ingest PDFs of hundreds of pages in a single call.