Distillation

Instead of learning from raw text alone, the student trains on outputs — answers, reasoning traces, or probability distributions — produced by a stronger teacher. The teacher's hard-won capability transfers into a model that is cheaper to serve, and the technique underlies most strong small models, including reasoning-distilled variants that inherit step-by-step problem solving.

Distillation is a primary engine of price decline: labs distill their own flagships into cheap tiers, and open-weights teams distill from powerful open teachers, so last year's frontier performance keeps reappearing at a fraction of the per-token price.

Distilling from proprietary APIs against the provider's terms of service is a recurring industry controversy, and providers increasingly monitor for extraction-scale usage patterns.