Batch API

Instead of calling the model synchronously and waiting for each response, you upload a file of requests and collect the results when the job completes. Providers commit to a completion window, most commonly 24 hours, and price batched tokens at roughly half the interactive rate.

The discount exists because batch traffic lets providers fill idle capacity and schedule work flexibly. For the customer, the tradeoff is pure latency: nothing about model quality or token accounting changes.

Batch processing suits any workload that is not user-facing in real time: bulk classification and extraction, embedding backfills, synthetic data generation, evaluation runs, and content pipelines. Routing eligible traffic to batch endpoints is often the single easiest way to cut a large inference bill.