Small language model (SLM)

There is no strict cutoff, but the label usually covers models small enough to run on a single consumer GPU or a phone. Model families like Phi, Gemma, and the smaller Llama and Qwen sizes made the category credible by matching much larger models from previous generations, largely through better data curation and distillation from stronger teachers.

SLMs excel at high-volume, well-scoped tasks: classification, routing, extraction, autocomplete, and moderation. Served via API they cost a small fraction of frontier rates per token, and self-hosted or on-device they can eliminate per-token fees entirely while keeping data local.

The practical pattern is tiering: let an SLM handle the bulk of predictable traffic and escalate ambiguous or hard cases to a larger model, cutting blended cost without a visible quality drop.