Fine-tuning

Fine-tuning continues training from a pre-trained checkpoint on a focused dataset, typically hundreds to tens of thousands of examples. Full fine-tuning updates all weights, while parameter-efficient methods like LoRA train small adapter layers instead, cutting cost dramatically and dominating open-weights practice.

Proprietary providers offer fine-tuning as a service: you pay per training token, and the resulting custom model often bills at a higher per-token rate than the stock version. With open weights you fine-tune on your own or rented hardware and pay no premium at serving time.

Fine-tuning shines for consistent format, tone, and domain-specific behavior, and it can let a small tuned model replace a large general one at far lower cost per request. It is the wrong tool for injecting fresh factual knowledge, where retrieval-augmented generation is usually cheaper and easier to update.