Reference · AI Landscape
AI & LLM Glossary
63 termsPlain-language definitions of the terms that matter when you evaluate AI models and their costs, from tokens and context windows to benchmarks, agents, and the chips that run them. Every definition links to the related concepts, models, and providers on Tokenando. Explore the AI landscape →
Pricing & Billing
14 termsAPI providerAn API provider is a company that serves AI models over an API, either its own proprietary models or open-weights models it hosts for customers.Batch APIA Batch API processes large sets of requests asynchronously, typically within 24 hours, in exchange for a discount, commonly 50 percent off standard rates.Blended costBlended cost is the effective price per token of a real workload, combining input and output rates weighted by the actual mix of tokens consumed.Context windowThe context window is the maximum number of tokens a model can handle in a single request, covering the input prompt and generated output combined.Free tierA free tier is a no-cost allowance of API usage or chat access that providers offer to attract developers, usually with strict rate and volume limits.Input tokensInput tokens are the tokens you send to a model (prompt, system instructions, documents, tool definitions, and chat history) billed at the input rate.Max output tokensMax output tokens is the upper limit on how many tokens a model may generate in one response, set by the model and adjustable per request.Model routerA model router is a service or layer that directs each request to one of several AI models based on cost, capability, latency, or availability.Output tokensOutput tokens are the tokens a model generates in its response, usually billed at a higher rate than input, often three to five times more per token.Price per million tokensPrice per million tokens is the standard unit of AI API pricing, quoted separately for input and output, for example, $3 per 1M input tokens.Prompt cachingPrompt caching lets a provider reuse computation for a repeated prompt prefix, discounting those input tokens (often by 50 to 90 percent) on later requests.Rate limitA rate limit caps how many requests or tokens an API customer can send per minute or day, protecting provider capacity and tiering access by spend.TokenA token is the basic unit of text an AI model reads and generates, roughly 4 characters or three-quarters of a word in English.TokenizationTokenization is the process of splitting text into tokens, the sub-word units a language model actually processes and providers bill for.
Models & Architecture
15 termsDistillationDistillation trains a smaller student model to imitate a larger teacher model's outputs, preserving much of its capability at far lower cost.Embedding modelAn embedding model converts text into a numeric vector that captures its meaning, enabling semantic search, clustering, and RAG retrieval.Foundation modelA foundation model is a large model pre-trained on broad data that serves as a base for many downstream tasks through prompting or fine-tuning.Frontier modelA frontier model is one of the most capable AI models available at a given time, pushing the state of the art in reasoning, coding, and knowledge.Large language model (LLM)A large language model (LLM) is a neural network trained on vast text corpora to predict tokens, enabling it to generate and reason over language.Mixture of Experts (MoE)Mixture of Experts (MoE) is an architecture that routes each token through a small subset of specialised sub-networks, cutting compute per token.Model deprecationModel deprecation is a provider's scheduled retirement of an API model, requiring users to migrate to newer versions before access is shut down.Model familyA model family is a set of related models sharing architecture and branding but differing in size, price, and capability: like Claude, GPT, or Gemini.Multimodal modelA multimodal model accepts or produces more than one type of data (such as text, images, audio, or video) within a single model.Open weightsOpen weights means a model's trained parameters are publicly downloadable, so anyone can run, fine-tune, or self-host it under its license terms.Parameter countParameter count is the number of learned weights in a model, a rough proxy for capability and for the compute needed to train and run it.Proprietary modelA proprietary model is available only through its creator's API or licensed platforms; the weights are never released for download or self-hosting.QuantizationQuantization reduces the numeric precision of a model's weights (for example from 16-bit to 4-bit) shrinking memory needs and speeding up inference.Reasoning modelA reasoning model spends extra compute thinking step by step before answering, trading higher latency and output-token cost for better accuracy.Small language model (SLM)A small language model (SLM) has relatively few parameters (often under about 15 billion) making it cheap to run, fast, and viable on-device.
Training & Alignment
7 termsAlignmentAlignment is the effort to make AI systems reliably pursue intended goals and behave safely, honestly, and helpfully, even in novel situations.Fine-tuningFine-tuning further trains an existing model on your own examples so it learns a task, style, or domain better than prompting alone can achieve.Instruction tuningInstruction tuning fine-tunes a base model on instruction-response examples so it follows user directions instead of merely continuing text.Pre-trainingPre-training is the initial, compute-intensive phase where a model learns language and world knowledge by predicting tokens across trillions of words.RLHFRLHF (reinforcement learning from human feedback) trains a model against human preference ratings so its answers become more helpful and safe.Scaling lawsScaling laws are empirical rules showing that model performance improves predictably as compute, data, and parameters are increased together.Synthetic dataSynthetic data is training data generated by AI models rather than collected from humans, used to expand or refine datasets for model training.
Inference & Prompting
9 termsHallucinationA hallucination is a confident but false or fabricated statement produced by an AI model, one of the main reliability risks in LLM applications.InferenceInference is running a trained model to produce outputs, every API call, chat reply, and agent step is inference, and it is what API pricing bills.LatencyLatency is how long a model takes to respond, commonly measured as time to first token (TTFT) and time per output token thereafter.Prompt engineeringPrompt engineering is the practice of designing model inputs (wording, structure, examples, and context) to get reliable, high-quality outputs.StreamingStreaming returns a model's output token by token as it is generated, instead of waiting for the full response, improving perceived speed.Structured outputStructured output constrains a model to produce machine-readable formats like JSON matching a schema, enabling reliable integration with software.System promptA system prompt is an instruction block that sets a model's role, rules, and context before any user messages, steering all of its responses.TemperatureTemperature is a sampling parameter controlling output randomness: low values make responses more deterministic, high values more varied and creative.ThroughputThroughput is the rate at which a model or provider processes tokens, typically measured in tokens per second for a request or across a whole system.
Benchmarks & Evaluation
7 termsBenchmarkA benchmark is a standardised test set used to measure and compare AI model capabilities, from general knowledge to coding and reasoning.EvalsEvals are automated tests that measure an AI system's quality on your own tasks, catching regressions when models, prompts, or providers change.GPQAGPQA is a graduate-level, Google-proof science benchmark whose questions stump even skilled humans with web access, testing deep scientific reasoning.HumanEvalHumanEval is an OpenAI benchmark of 164 hand-written Python problems that measures whether a model can generate functionally correct code.LMArena (Chatbot Arena)LMArena, formerly Chatbot Arena, ranks AI models by Elo-style scores from blind, head-to-head human votes on real user prompts.MMLUMMLU (Massive Multitask Language Understanding) is a benchmark of multiple-choice questions across 57 subjects, long used to gauge LLM knowledge.SWE-benchSWE-bench measures whether AI models can resolve real GitHub issues from popular Python repositories, the standard benchmark for agentic coding.
Agents & Tooling
5 termsAI agentAn AI agent is a system in which a model plans and executes multi-step tasks by calling tools, reading results, and iterating toward a goal.Computer useComputer use is an AI capability where a model operates a real computer interface: viewing the screen and issuing clicks, keystrokes, and scrolls.Function calling (tool use)Function calling lets a model request that your code run a specific function with structured arguments, then use the result to complete its answer.Model Context Protocol (MCP)The Model Context Protocol (MCP) is an open standard, introduced by Anthropic in 2024, for connecting AI models to external tools and data sources.Retrieval-Augmented Generation (RAG)Retrieval-Augmented Generation (RAG) retrieves relevant documents at query time and adds them to the prompt, grounding answers in your own data.
Chips & Infrastructure
6 termsAI acceleratorAn AI accelerator is any chip purpose-built to speed up neural network workloads, including GPUs, TPUs, and specialised inference processors.AI datacenterAn AI datacenter is a facility packed with accelerators, high-speed networking, and advanced cooling, built to train and serve AI models at scale.GPUA GPU (graphics processing unit) is the parallel processor powering most AI training and inference; Nvidia data-centre GPUs dominate the market.HyperscalerA hyperscaler is a cloud giant (AWS, Microsoft Azure, or Google Cloud) operating global datacenter fleets that host much of the world's AI compute.Inference providerAn inference provider is a company that hosts open-weights models and sells API access, competing on price per token, speed, and reliability.TPUA TPU (Tensor Processing Unit) is Google's custom AI chip, used to train and serve its own models and rented to customers through Google Cloud.