AI Glossary
Plain-English explanations of LLM and AI terms that matter for developers. No fluff, no hype - just what you need to understand to build with AI.
Core Concepts
Agentic AI
AI systems that autonomously execute multi-step tasks by deciding which tools to use, in what order, and when to stop.
Read more →Context Window
The maximum number of tokens an LLM can process in a single inference call, including both input (prompt, context, history) and output (generated response).
Read more →Inference
The process of running a trained model on input data to generate outputs, also called the serving phase as opposed to training.
Read more →Tokenization
The process of converting text into a sequence of integers that an LLM can process, where each integer represents a token - typically a subword fragment, full word, or punctuation character.
Read more →Architecture
Embeddings
Numerical vector representations of text, images or other data that capture meaning, so similar content ends up close together in vector space.
Read more →MCP (Model Context Protocol)
An open protocol, created by Anthropic and now hosted by the Linux Foundation's Agentic AI Foundation, that standardizes how AI applications connect to external tools, data sources and services.
Read more →MoE (Mixture of Experts)
A neural network architecture where many specialized sub-networks (experts) exist within a single model, but a router activates only a few of them for each token, enabling large total parameter counts with much lower per-token compute.
Read more →Multimodal
A model that accepts or produces more than one type of data, such as text, images, audio or video. Most current frontier models take text and images in but answer only in text; speech, image and video output usually come from separate specialized models.
Read more →RAG (Retrieval-Augmented Generation)
A technique that improves LLM answers by retrieving relevant passages from an external knowledge base and adding them to the prompt before generation, reducing hallucinations and giving the model access to information beyond its training data.
Read more →Vector Database
A database optimized for storing and querying high-dimensional embedding vectors, enabling fast approximate nearest-neighbor search.
Read more →Training
Distillation
Training a smaller student model to reproduce a larger teacher model's behavior by learning from the teacher's outputs or output probabilities rather than ground-truth labels alone.
Read more →Fine-tuning
Continued training of a pre-trained model on a smaller, task-specific dataset to adapt its weights for a particular domain, task, or output format.
Read more →LoRA (Low-Rank Adaptation)
A parameter-efficient fine-tuning method that freezes a model's weights and trains small low-rank matrices added to selected layers, cutting trainable parameters by up to 10,000x (shown on GPT-3 175B in the original paper) with quality close to full fine-tuning on many tasks.
Read more →RLHF (Reinforcement Learning from Human Feedback)
A post-training method in which people compare model outputs, a reward model learns their preferences, and reinforcement learning tunes the model toward the preferred answers. Labs now pair it with AI feedback and reinforcement learning on automatically checkable tasks.
Read more →Inference
JSON Mode
An API setting that constrains an LLM to output valid JSON without enforcing a specific schema.
Read more →KV Cache
A store of the attention key and value vectors for tokens a model has already processed, so each new token can be generated without recomputing the whole context.
Read more →Latency vs Throughput
Latency is how long a single request takes, usually split into time to first token and output speed; throughput is how much work a system completes per second across all concurrent requests.
Read more →Structured Output
Constraining an LLM to return output that matches a defined format, usually a JSON Schema, so code can parse it directly instead of extracting values from free text.
Read more →Temperature
A sampling parameter that controls randomness in an LLM's output by scaling the probability distribution of tokens before sampling - lower values produce more deterministic results, higher values produce more varied results.
Read more →Top-p (Nucleus Sampling)
A sampling strategy that selects tokens by including the most probable candidates until their cumulative probability reaches a threshold p, allowing the number of candidates to vary based on the model's confidence.
Read more →Prompting
Chain-of-Thought (CoT)
A prompting technique where a model is instructed to produce intermediate reasoning steps before generating a final answer, improving performance on complex tasks like math, logic, and code generation.
Read more →Prompt Injection
An attack where malicious instructions embedded in external content attempt to override an LLM's system prompt or alter its behavior.
Read more →System Prompt
Instructions provided to an LLM at the start of a conversation that define its persona, behavior, response format, and constraints.
Read more →Tool Use / Function Calling
The ability of an LLM to request execution of external functions, APIs, or tools by returning a structured call that matches a declared schema, so an application or the provider's servers can run it and return the result.
Read more →