Streaming

With streaming enabled, the API delivers tokens over a persistent connection — typically server-sent events — the moment each is produced. The user sees text appear within a fraction of the total generation time, which is why every serious chat interface streams: perceived latency collapses even though total generation time is unchanged.

Billing is identical to non-streaming; the same tokens cost the same amount. The engineering differences lie elsewhere: handling partial JSON when streaming structured output, rendering incremental tool-call arguments, and accounting for usage totals that arrive only at stream end.

Streaming pairs naturally with long-running requests from reasoning models and agents, where providers also stream thinking summaries and intermediate events so applications can show progress instead of a spinner.