Input tokens

Everything the model reads counts as input: the user message, the system prompt, few-shot examples, retrieved documents, tool and function definitions, and the entire prior conversation in a chat. In multi-turn applications the history is resent on every request, so input token usage compounds as conversations grow.

Input tokens are almost always priced lower than output tokens, typically by a factor of three to five, because processing a prompt is computationally cheaper than generating a response. Even so, input usually dominates the bill for long-context workloads like document analysis, RAG, and agents, where prompts run to tens of thousands of tokens against short answers.

Prompt caching can discount repeated input heavily, and trimming history, deduplicating context, and shortening tool definitions are among the most effective levers for cutting API spend.