Context window

Everything the model works with must fit inside the window: system prompt, conversation history, documents, tool results, and the response being generated. Early GPT-3-era models offered around 4,000 tokens; by the mid-2020s frontier models routinely offered 128,000 to 1 million tokens, enough for entire codebases or books.

A large window is not free capacity. Every token in context is billed as input on every request, and some providers charge a higher rate once a prompt crosses a size threshold. Filling a million-token window on each call is one of the fastest ways to run up an API bill.

Long context also has quality limits: models can attend less reliably to material buried in the middle of enormous prompts. Retrieval-augmented generation, which selects only relevant passages, is often both cheaper and more accurate than stuffing the full corpus into every request.