BETA
A token is the basic unit of text an AI model reads and generates — roughly 4 characters or three-quarters of a word in English.
Language models do not process raw characters or whole words. Before a model sees your text, it is split into tokens: sub-word chunks drawn from a fixed vocabulary. In English, one token averages about 4 characters, so 1,000 tokens correspond to roughly 750 words.
Tokens are the billing unit of the AI API economy. Providers quote prices per million tokens, with separate rates for input and output, so estimating how much text a workload converts into tokens is the first step in estimating its cost.
Token counts vary by language and content type. Source code, non-English languages, and dense formats like JSON often consume more tokens per word than plain English prose, which directly raises the cost of sending the same logical payload.
Last revised 2026-07-05 · All glossary terms → · Live AI model pricing →