AI accelerator

The umbrella covers Nvidia and AMD GPUs, Google TPUs, Amazon's Trainium and Inferentia, and a wave of specialist designs: Groq's deterministic LPUs and Cerebras's wafer-scale engines both target inference speed, advertising output rates far beyond GPU-based serving on supported models. What unites them is optimization for tensor math at low precision with extreme memory bandwidth.

Training and inference reward different designs. Training needs massive clusters with fast interconnects and high-precision support; inference rewards latency, throughput per watt, and cost per token, which is why inference-first architectures can beat general-purpose GPUs on serving economics for specific models.

For AI buyers the chip layer is invisible but priced in: which accelerators a provider runs shapes its token prices and speed tiers, and the steady arrival of credible Nvidia alternatives is one of the structural forces behind falling inference costs.