AI Context Windows Explained: Tokens, Limits and Real-World Use

A context window is one of the most important specifications attached to a modern AI model, but it is also one of the easiest to misunderstand.

When a model advertises a 128K, 200K, or 1-million-token context window, that number indicates how much information the model can work with during a request. Depending on the model, the total can include your prompt, previous conversation turns, uploaded material, tool results, generated output and even tokens used internally for reasoning.

A larger context window can enable analysis of long documents, support lengthy conversations, or work across large collections of code. It does not mean the model permanently remembers that information, nor does it guarantee perfect recall of everything placed within the window.

That distinction becomes increasingly important as leading AI models expand their context windows to roughly 1 million tokens.

What Is an AI Context Window?

The simplest way to think about a context window is as the model’s working space for the current task.

Anthropic describes context as a form of working memory that contains the information available to the model while it produces a response. Google uses a similar short-term memory analogy for Gemini. It is separate from the enormous dataset used to train the model.

Suppose you start a conversation by providing a 20-page report and asking for a summary. The report, your instructions and the generated response all consume part of the available context.

If you then ask five follow-up questions, the conversation history may also be carried forward. The amount of context being processed therefore grows as the conversation continues.

Once the available context is exhausted, something has to change. Depending on the application and model, older information may be removed, summarised, or compacted, or the request may be rejected for exceeding the limit. OpenAI and Anthropic both provide mechanisms for managing conversations as they approach those limits.

What Is a Token?

AI models do not usually read text one word at a time. They process it as tokens.

A token may be a complete word, part of a word, punctuation or even a single character. The exact breakdown depends on the model’s tokeniser and the language being processed.

OpenAI provides a useful rule of thumb for English: one token is roughly 4 characters, or about 3/4 of a word. That means 100 tokens correspond to approximately 75 English words. The relationship is only an estimate, and token counts can vary significantly across languages and content types

Using that rough English conversion:

• 10,000 tokens can represent around 7,500 words.

• 100,000 tokens can represent around 75,000 words.

• 1 million tokens can represent roughly 750,000 words.

These figures should not be treated as exact document capacities. Code, numbers, punctuation, different languages and specialised formatting tokenise differently.

Google gives another sense of scale. Its long-context documentation says that 1 million tokens can accommodate roughly 50,000 lines of code, using 80 characters per line, or the equivalent of about eight average-length English novels.

Input Tokens and Output Tokens Are Different

The headline context number is not necessarily the amount of text you can paste into a prompt and still receive an equally large answer.

Models usually distinguish between input capacity and maximum output.

Input tokens can include your instructions, chat history, documents, images represented within the model’s context, tool results and other information provided to the model.

Output tokens are what the model generates in response.

Some reasoning models also use reasoning tokens as part of their processing budget. OpenAI’s documentation states that input, output and reasoning tokens can all count toward the model’s overall context window.

This is why a model with a 1-million-token context window does not necessarily produce a 1-million-token answer.

For example, OpenAI’s current GPT-5.6 Sol API model lists a 1,050,000-token context window but a maximum output of 128,000 tokens.

Google’s current Gemini 3.7 Flash lists an input token limit of 1,048,576 and an output token limit of 65,536.

Anthropic’s Claude Sonnet 5 has a 1-million-token context window and supports up to 128,000 output tokens per request.

The input and output limits therefore need to be considered separately when comparing models.

How Large Are Context Windows Now?

Context windows have expanded rapidly.

Older language models commonly worked within context sizes measured in thousands or tens of thousands of tokens. Google’s long-context documentation notes the progression from 8,000-token systems through 32,000 and 128,000-token models before million-token contexts became practical.

By August 2026, million-token contexts are available across several leading model families.

OpenAI lists 1,050,000 tokens for GPT-5.6 Sol, Terra and Luna in its current API model comparison.

Anthropic lists 1-million-token windows across its current Claude models, including Claude Opus 5 and Claude Sonnet 5. Google’s Gemini 3.7 Flash supports an input limit of 1,048,576 tokens.

Those numbers show how far the technology has moved, but comparing models only by their maximum context figure can be misleading.

A Bigger Context Window Does Not Mean Better Memory

A context window is not the same thing as persistent memory.

Context is the information available during a request or conversation. Persistent memory is information an AI product may save separately and make available in later conversations.

A model could therefore have an enormous context window while knowing nothing about a conversation you had with it last month unless the application retrieves that information and places it back into the active context.

There is another limitation: fitting information into a context window does not guarantee that the model will use every piece of that information equally well.

Anthropic explicitly warns that more context is not automatically better. Its documentation says accuracy and recall can degrade as the amount of context grows, a problem it refers to as “context rot.”

This is why a carefully selected 30,000-token prompt can sometimes be more useful than sending hundreds of thousands of loosely relevant tokens.

Long Documents Are an Obvious Use Case

Large context windows are particularly useful for document analysis.

Instead of breaking a report into dozens of fragments, a user may be able to place much more of the document into a single context and ask questions that require information from distant sections.

That can help with tasks such as:

• Comparing provisions across contracts.

• Reviewing lengthy technical documentation.

• Analysing research papers together.

• Finding inconsistencies across reports.

• Creating a summary based on an entire source collection.

• Tracing a topic through hundreds of pages.

There are still limits beyond the nominal token count. Anthropic, for example, notes that request-size limits can be reached before the token limit when a request includes a large number of images or PDF pages.

A “1 million-token model” should therefore not be interpreted as a promise that every application will accept any file containing fewer than 1 million tokens.

Context Windows Matter for Coding

Coding is another area where long context can make a noticeable difference.

A small context window may allow an AI coding assistant to inspect only the current file and a few neighbouring functions. A larger one can include multiple files, tests, configuration information,n and documentation within the same working context.

That can help when a change in one part of a codebase affects several other components.

Google estimates that 1 million tokens can represent about 50,000 lines of conventionally formatted code, although actual token consumption varies considerably by programming language and formatting.

Even then, feeding an entire repository into every request may not be the best approach. Modern coding systems often combine large context windows with search, indexing and retrieval so the model receives the parts of the repository most relevant to the current task.

Long Conversations Eventually Consume Context

Every turn in a normal AI conversation can add more material to the active context.

A later request may include the original system instructions, earlier user questions, previous model responses and the newest prompt. Tool results and documents can add even more. Anthropic notes that all of these elements can consume context capacity in its API.

This explains a behaviour users sometimes interpret as the AI forgetting.

Once a conversation becomes sufficiently long, the application may need to drop older information, summarise it or otherwise compress the conversation.

Both OpenAI and Anthropic now document approaches to compacting conversations to keep them running when their histories become too large for the active window.

The result is that an AI conversation can continue beyond a single context window, but the complete original history may not remain available word-for-word forever.

More Context Can Also Mean More Cost

For API users, long context has an economic cost.

Model providers generally charge according to token usage, with input and output often priced separately. Sending a large document repeatedly can therefore be substantially more expensive than sending only the passages necessary for each question.

Caching can reduce the cost of repeatedly processing the same material on supported platforms, but caching should not be confused with increasing the model’s context window.

Anthropic states that cached prompt tokens continue to occupy the context window even when caching changes how those tokens are billed.

Google likewise provides context caching to reduce the cost and processing overhead associated with repeatedly sending the same large input.

This matters for applications that repeatedly ask questions about the same book, manual, codebase or collection of documents.

Context Window vs RAG

A large context window does not eliminate the need for retrieval-augmented generation, commonly called RAG.

RAG searches an external knowledge source and retrieves a smaller set of relevant information before sending it to the model.

With a sufficiently large context window, developers could instead include a large amount of information directly in every prompt. In practice, that may increase latency, token costs, and the amount of irrelevant information.

Retrieval can keep the context focused.

The two approaches are therefore complementary. A large context window gives a system more room when a task genuinely requires broad information, while retrieval helps decide which information deserves that room.

How Much Context Do You Actually Need?

For most ordinary questions, very little.

A request to rewrite an email or explain a simple concept does not benefit much from hundreds of thousands of tokens of context.

Larger windows become useful when the model needs to connect information spread across a large body of material. Examples include reviewing a long legal agreement, analysing an entire research collection, working across a codebase, or maintaining state during a lengthy agent task.

The practical goal is not to fill the context window. It is to give the model enough relevant information to perform the task without burying useful details in unnecessary material.

What Context-Window Numbers Really Tell You

A context-window specification tells you the maximum working capacity available to a model under particular conditions. It does not tell you how intelligent the model is, how accurate it will be across that entire window or how much information an AI application will permanently remember.

When comparing models, look beyond the headline token number.

Check the maximum input and output sizes, reasoning-token behaviour, application-level file restrictions, price per token, and the model’s performance with long information.

An 11-million-token context window is genuinely useful when a task requires an enormous amount of material. For everyday AI use, however, relevant context remains more valuable than maximum context.