Tokens, context, window: what these words mean

You have surely seen these words in the settings of an AI assistant or in a model’s description: “token,” “context window,” “200K tokens.” They are not specialist details. They directly explain why AI “forgets” mid-conversation, why an answer stops in the middle of a sentence, and why you sometimes need to split a document before submitting it.

What is a token, concretely?

A token is the small fragment of text that the model reads and manipulates — neither quite a letter, nor quite a word. Language models do not process text word by word: they break it into fragments called tokens. A common word like “hello” is often a single token; a longer, rarer or compound word (like a surname or a technical term) can be worth two, three or more.

In English, a simple order of magnitude holds: about 750 words roughly represent 1,000 tokens. It is not an exact rule — punctuation, numbers and rare words shift the count — but it is enough to find your bearings.

Why this matters for you: tokens are the unit tools use to measure the length of a conversation, and often to bill usage of paid plans — indeed, it is in dollars per million tokens that model price cuts play out. It is also the unit behind the next limit, the most concrete one day to day.

What is the context window?

The context window is the total amount of text — your message, the conversation history, a pasted document — that the model can “look at” at the same time to produce its answer. It is measured in tokens: a model with a 200,000-token window can process at once the equivalent of about 150,000 words, roughly a medium-sized book.

Each model has its own, and it varies by tool and by plan. It includes everything that enters the exchange: your question, attached documents, and the history of the ongoing conversation. Once this limit is reached, something has to give way.

Why AI seems to “forget” along the way

A long conversation with an AI assistant eventually fills its context window. When the limit approaches, two things can happen depending on the tool: either the conversation refuses to accept a new message until you have shortened the exchange, or the very first messages gradually leave the range the model can still consult — a bit like a stack of notes from which you remove the bottom to add to the top.

This is what explains that frustrating impression: you had given important context at the very start of the conversation (your job, a constraint, an example), and the AI seems to have “forgotten” it twenty messages later. It has not forgotten it in the human sense: the information has simply left what it can still read.

The useful reflex: on a long, important exchange, restate the key context every few dozen messages, or better, open a new conversation dedicated to each new topic rather than piling everything into the same thread.

Why you sometimes need to split a document

Submitting a 300-page report all at once can exceed the tool’s context window, or approach it to the point of degrading answer quality — even large-window models often handle information in the middle of a very long text less well than at the start or the end.

Two practical reflexes:

• Split by section rather than by an arbitrary page count: a chapter, a logical part, a coherent whole. AI keeps the thread better on a piece that makes sense.
• Summarise as you go if you have to process a document in several passes: ask for a synthesis of each part, then submit those syntheses together for an overview. This is the method detailed in our guide to summarising a long document without betraying it.

Wide window, smart model: not the same thing

A common mistake: confusing the size of the context window with the quality of the model. A large window means the tool can see more text at once — not that it reasons better on that text. A model with a more modest but well-designed window can stay more precise and more relevant on a short text than a giant-window but less rigorous model.

Concretely, when choosing a tool, the size of the context window is only a decisive criterion if your use is precisely to process very long documents in one go. For everyday writing, daily exchanges or task automation, other criteria matter more — see our comparison ChatGPT, Claude or Gemini: which for which use.

The table to remember

Word	What it is	What it changes for you
Token	Fragment of text read by the model (≈ 1 word or less)	Unit for measuring length and often price
Context window	Total amount of text the model can process at once	Determines how much conversation or document AI “sees”
Conversation memory	What stays visible in the window over the exchanges	Explains why AI seems to forget an earlier point

Key takeaway

These three words are not gratuitous jargon: they explain behaviours you already observe without necessarily naming them. Remember the essential — a token is a small fragment of text, the context window is the total amount the model can process at once, and a conversation or document that is too long eventually falls out of it. On an important exchange, restate the context regularly; on a large document, split by section rather than hoping everything fits at once.