Multimodal model

Most frontier models are now natively multimodal on the input side, accepting images and often audio or video alongside text. Output multimodality — generating images, speech, or video — is sometimes built into the same model and sometimes served by dedicated companion models.

Non-text inputs are billed by converting them into token equivalents: an image is sliced into patches or tiles that count as input tokens, and audio is metered per second or per token depending on the provider. A single high-resolution image can cost as much as several pages of text, which matters for pipelines that process screenshots or documents at volume.

Multimodality changes workload design: tasks that once required a separate OCR or speech-to-text step can go straight into the model, trading pipeline complexity for token spend.