Model router

Routers come in two flavors. Aggregator gateways such as OpenRouter expose hundreds of models from many providers behind one API and one bill, letting developers switch models by changing a string. Smart routers go further and choose the model per request, sending easy queries to cheap models and hard ones to frontier models.

The economic case is strong: in most applications a large share of requests can be served by a model an order of magnitude cheaper than the flagship without measurable quality loss. Routing also enables automatic failover when a provider has an outage or a rate limit is hit.

The tradeoffs are an extra network hop, a middleman fee or markup on some platforms, and another party in the data path, which security reviews need to account for. Routed prices for open models can also differ from the official rates a model creator publishes.