AI & Infrastructure Glossary

A model is the trained neural network itself: weights that map input tokens to output token probabilities. It is a static artifact: it predicts, but it does not act on its own. GPT, Claude, and Llama are models. Contrast an agent, which is a model placed inside an action loop.

An LLM (Large Language Model) is a model trained on massive text to predict tokens, large enough to exhibit general language ability: writing, reasoning, coding, and translation. "Large" refers to parameter count, which ranges from billions to trillions. In most AI discussion, "model" refers to an LLM.

An agent is a model wrapped in a loop that can take actions (call tools, read results, decide the next step) toward a goal without a human driving each turn. Its defining feature is autonomy over multiple steps: perceive, decide, act, observe, repeat. Industry usage of the term is not fully settled; different practitioners draw the line at different degrees of autonomy.

Agentic is an adjective describing how much autonomous, multi-step, tool-using behavior a system exhibits. It is a spectrum rather than a binary, leaning toward self-directed action instead of single-shot response. The term is frequently overused in marketing, and has no single agreed threshold.

An orchestrator is the control layer that coordinates multiple agents, models, or tool calls, routing work, sequencing steps, managing state, and deciding who does what. It can be code (deterministic control flow) or another model acting as a router. It is the "manager" above the "workers."

Versioning is the practice of tracking distinct iterations of a model, prompt, dataset, or spec so behavior is reproducible and changes are auditable. It matters more in AI than in traditional software because outputs are non-deterministic, and a silent change to weights or a prompt can shift behavior invisibly.

A registry is a catalog of discoverable, installable components (models, tools, MCP servers, plugins, or agents) with metadata for lookup and install. It is the "app store" pattern applied to agent parts.

A spec is a declarative definition of what something is or should do, kept separate from the code that runs it: a tool's input schema, an agent's config, an API contract, or an eval's expected behavior. Increasingly it means the structured description a model reads to know how to use a capability.

A token is the atomic unit of text a model processes: a sub-word chunk of roughly four characters, or about 0.75 words in English. Models do not see letters or words; they see token IDs. Context limits, pricing, and speed are all measured in tokens.

Tools are functions an agent can call to affect or read the world beyond generating text: search, run code, query a database, send a message. Each is defined by a name, description, and input schema; the model chooses when to invoke a tool and reads the result back. Tools are what turn a model into an agent.

Function calling is the mechanism by which a model invokes a tool: it emits a structured request (tool name plus arguments matching the tool's schema), the harness runs the real function, and the result is fed back into the context. The model never executes anything itself: it only outputs the intent to call, and the harness performs the call. This is the plumbing beneath Tools and the concrete meaning of an action.

Connections are authenticated links between an agent and external systems or data (accounts, apps, APIs), the plumbing that grants a tool access to a specific user's Slack, email, or files. Tools are the capability; connections are the authorized access the capability runs through.

Skills are packaged, reusable bundles of instructions (and sometimes scripts or resources) that teach an agent how to do a specific task well. They are loaded on demand so the model does not carry every procedure in context at once. They are modular expertise the agent pulls in when relevant.

Training is the process of creating a model by adjusting its weights over massive data until its predictions improve. It is expensive, done once per version, and offline. Training is where the model learns; nothing about the world after training day is contained in it.

Inference is running the finished model to produce an output: a prompt goes in, tokens come out. It is cheap per call, done constantly, and live. The distinction between training and inference is the single most useful one in AI: training builds the model, inference uses it. Everything a user experiences when interacting with a deployed model is inference.

Weights (or parameters) are the numbers inside the model that were tuned during training, billions of them. "Parameters" refers to the count (a "70B model" has 70 billion); "weights" refers to the values themselves. They are the model; everything it knows is encoded in them.

Pre-training is the first and largest training phase: predicting the next token across a huge swath of text. It produces a "base model" that is knowledgeable but unruly: good at completion, poor at following instructions.

Fine-tuning is further training of a pre-trained model on a narrower dataset to specialize its behavior. It is far cheaper than pre-training. Fine-tuning is one of three ways to adapt a model to a need; the others are prompting and RAG. It changes what the model is.

RLHF is the tuning step that turns a raw base model into a helpful assistant: humans rank outputs, and the model is trained to prefer the ranked-good ones. It is where "be useful, honest, harmless" gets installed, and the reason an assistant answers questions rather than merely autocompleting text.

A dataset is the corpus a model is trained or fine-tuned on. Its quality, size, and cutoff date bound what the model can know and how it behaves. A model's knowledge cutoff is a property of its dataset.

Compute is the raw processing power that training and inference run on. A GPU (graphics processing unit) is the workhorse chip: built for graphics, but its massively parallel math is exactly what neural networks need. A TPU is a custom, purpose-built equivalent for the same job. "Accelerator" is the generic term; "compute" is the resource itself, scarce and expensive, and the reason chipmakers sit near the center of the industry. More compute enables bigger models, more training, and faster inference.

This is the distinction of whether a model's trained weights are published. Open-weight models can be downloaded, run, fine-tuned, and self-hosted by anyone, though open-weight is not the same as fully open-source, since training data and code usually stay private. Closed or proprietary models never release their weights, and are reachable only through an API. This is a central strategic fault line of the industry: open ecosystems versus API-gated access.

A frontier model is one at the leading edge of capability: the largest, most capable generation available at a given moment. A frontier lab is an organization that builds such models. The term carries both the cutting-edge capability and the heavy compute and safety stakes that come with operating at that edge.

The context window is the finite span of tokens a model can consider at once: everything it "sees" on a given turn, including the system prompt, conversation, tool results, and files. It is measured in tokens and is the model's entire working reality for that turn; nothing outside it exists to the model.

A prompt is the input given to the model on a turn. Loosely, it is the user's message; precisely, it is the entire assembled context the model receives.

A system prompt is the standing set of instructions placed at the top of the context that shapes the model's role, tone, and rules for a whole session. It is set by the developer, not the end user. It is the operating instructions the model reads before the first user message.

In-context learning is the model's ability to pick up a new pattern or task from examples in the prompt, with no training or weight change. Shown three examples of a format, it follows the format. It is adaptation at inference time, notable because nothing is "learned" in the training sense.

Context rot is the decline in output quality as the context window fills: the model loses track, over-weights recent tokens, or contradicts earlier state. It is why long sessions drift, and why managing what is in the window matters as much as what the model knows.

An API is a contract for how two pieces of software communicate: a defined set of endpoints, inputs, and outputs one program exposes so another can call it. It is general-purpose and ubiquitous. Each API is bespoke, with its own auth, shapes, and docs, so connecting an app to 100 APIs means writing 100 integrations. In one line: an API is how software exposes a capability.

MCP (Model Context Protocol) is an open standard, introduced by Anthropic in late 2024, for how an AI model or agent connects to tools and data. It is a single, uniform protocol that wraps underlying capabilities so any MCP-compatible agent can discover and use any MCP server without custom glue: to connect an agent to 100 MCP servers, you speak one protocol 100 times. MCP has two sides: a client (the agent or app requesting things) and a server (the wrapper exposing a tool or data source). A server can expose tools (actions the model can call), resources (data it can read), and prompts (reusable templates), and the two sides communicate over a defined transport: local process I/O for a server on the same machine, HTTP for a remote one. Because the interface is standardized, a server built once works with any MCP-compatible client.

MCP is not an alternative to APIs; it sits above them. An MCP server is usually a standardized wrapper around one or more existing APIs (or files, or a database). The API is the actual capability; MCP is the common language that lets a model use that capability without knowing its specific shape. By analogy, APIs are individual power outlets, each region with its own plug and voltage, while MCP is the universal adapter that fits them all. APIs were designed for programmers who read docs and hand-write integration code; MCP is designed for models, which must discover capabilities at runtime, read machine-readable descriptions, and call them uniformly. This turns the M×N integration problem (M agents times N tools) into M+N (each speaks MCP once).

A harness is the code scaffolding around a model that makes it an agent: the loop that feeds it context, exposes its tools, parses its outputs, runs the actions, and feeds results back. The model is the engine; the harness is the chassis, wiring, and pedals. (In broader ML usage, a harness is any rig that runs a model against inputs, such as an "eval harness.") A key principle: the same model in two different harnesses behaves very differently. Much of what users attribute to "the agent" is actually the harness.

A workspace is the environment an agent operates in: its filesystem, working directory, and the state that persists across steps within a session. It is where the agent reads, writes, runs code, and stashes intermediate work. A common form is an isolated cloud container: a private Linux box with a shell and disk that lasts the session and is then reclaimed.

A shell is the command-line interpreter (such as bash or zsh) that takes typed commands and runs them against the operating system. For an agent it is the most general-purpose tool available: one interface to the whole machine, to install packages, run scripts, move files, and call other programs. It subsumes a hundred narrower tools, which is what makes it both powerful and worth constraining.

A loop is the repeating cycle at the heart of every agent: take in context, decide on an action, execute it, observe the result, feed that back in, and repeat until the goal is met or a stop condition is hit. In short, "agent" means "a model put in a loop."

Actions are the concrete moves an agent makes on each pass of the loop: calling a tool, running a shell command, writing a file, sending a message. They are the output of "decide" and the input to "observe." A model generates text; actions are what that text becomes when the harness interprets it as a tool call and runs it.

Memory is how an agent retains information beyond the immediate context window. Models have no built-in memory; any persistence is engineered around them. There are two kinds. Short-term memory is what is in the current context (the conversation and results so far), which is fast, rich, and wiped when the session ends. Long-term memory is state deliberately written somewhere durable (a file, a database, a memory store) so a future session can read it back; it survives the session but must be explicitly saved.

A session is one continuous run of an agent: a bounded stretch with its own context, workspace, and state, from start until it ends or is reclaimed. It is the unit across which short-term memory lives and dies. Two sessions share nothing unless something was written to long-term memory between them.

The pieces fit together as a chain: a model runs inside a harness that drives a loop; on each turn the model chooses actions that run in the workspace via tools like the shell; memory is what survives each step, and the session is what bounds the whole run.

An embedding is a vector, a list of numbers, that represents the meaning of a piece of text, arranged so that similar meanings land near each other in vector space. It is the trick that lets a machine compare text by meaning rather than by exact words.

A vector database is a store built to hold embeddings and answer "what is most similar to this?" quickly. It is the retrieval backend for meaning-based search.

Retrieval is fetching the most relevant chunks of external knowledge, usually via embeddings and a vector database, to inject into the context at inference time. It is the step of getting the right pages onto the desk before the model answers.

RAG (Retrieval-Augmented Generation) is the pattern of retrieving relevant documents and placing them in the context so the model answers from them rather than from memory. It is how you give a model current, private, or specialized knowledge without retraining it, and is one of the three ways to adapt a model (alongside fine-tuning and prompting).

Chunking is splitting documents into passages small enough to embed and retrieve. It is unglamorous but decisive: chunk badly and retrieval returns garbage.

There are three ways to adapt a model to a need, from lightest to heaviest. Prompting puts instructions and examples in the context, with no training. RAG retrieves external knowledge into the context at query time, also with no training. Fine-tuning retrains the weights on new data. Prompting and RAG change what the model sees; fine-tuning changes what the model is.

Artifacts are the durable work product a user actually wanted: the files, report, code, spreadsheet, or trained model, the point of the session. This is the strict meaning of "artifact," and the best single umbrella word for what a session leaves behind.

State (in the sense of durable residue) is the memory written for the agent's own benefit across sessions: a memory file, a context file, a database of what has been done, a scratch log. It is not the deliverable. It is the substrate that lets the next run pick up where the last left off. It is long-term memory viewed as a thing on disk.

A trace is the record of how work was done: the transcript, logs, and run history: every turn, tool call, and result, in order. It is not what was produced but the auditable path of how it got produced, and it is what you read to debug or verify the work.

Exhaust is the accumulated non-deliverable buildup that nobody pruned: stale caches, dead branches, half-finished scratch files, an unmaintained memory store. It is the part people forget to clean up, and after enough turns it is often the largest pile.

A hallucination is a model stating something false with full fluency and confidence, inventing a citation, a fact, or an API that does not exist. It is not lying, since there is no intent; it is a byproduct of a system built to produce plausible text, which is not the same as true text. It is the reason well-designed systems show their work and cite sources rather than asking for blind trust.

An eval (evaluation) is a structured test of model or agent behavior against known-correct answers or graded criteria. It is how you measure whether a change actually helped instead of guessing. It is the scientific method for non-deterministic systems.

A benchmark is a standardized, shared eval used to compare models against each other. It is useful for rough ranking but is gameable and often unrepresentative of real use, and should be treated with skepticism.

Alignment is the degree to which a model's behavior matches human intent and values. It is the broad goal; RLHF and guardrails are among the means of pursuing it.

Guardrails are the constraints layered on a model to keep outputs within bounds: refusals, filters, policy checks, permission gates. They are not part of the model's intelligence. They are a fence around it.

A jailbreak is a prompt crafted to bypass a model's guardrails and elicit behavior it is meant to refuse. It is the adversarial counterpart to guardrails, and specifically describes a user attacking the model directly.

Prompt injection is hostile instructions hidden in content the agent reads (a web page, a file, a tool result) rather than in what the user types, designed to hijack the agent into doing an attacker's bidding. It is the live threat model for any agent with tools: the danger is not only what the user asks, but what the data says. It is distinct from a jailbreak (a user attacking the model directly); an injection is a third party attacking through the content the model consumes. This is why an agent that reaches web pages and files should treat what it reads as untrusted.

Sampling is choosing each next token from the model's probability distribution. Because it is a sample rather than a fixed pick, the same prompt can yield different outputs. That is the root of the term "non-deterministic."

Temperature is the knob controlling how random sampling is. Low temperature (near zero) makes the model pick the highest-probability token almost every time, focused and repeatable. High temperature makes it take more chances, varied, creative, and riskier. It is the dial between deterministic and surprising.

Modality is a type of data a model can take in or produce: text, image, audio, or video. A multimodal model handles more than one, for example reading an image and writing about it.

Reasoning is a model working through a problem in intermediate steps before committing to a final answer, rather than answering in one shot. "Chain-of-thought" is the visible step-by-step trace. Modern reasoning models are trained to do this extensively, to "think" first, which sharply improves math, coding, and logic at the cost of more tokens and latency.

Test-time compute is spending more computation at inference (longer reasoning, multiple attempts, self-checking) to get a better answer, instead of relying only on a bigger pre-trained model. It is the insight behind reasoning models: inference time can be traded for quality. It marks a shift from "all the intelligence is baked in at training" to "some is bought at the moment of answering."

Infrastructure as Code is describing servers, networks, and databases in version-controlled text files instead of clicking through a console by hand. You declare what the infrastructure should look like, and a tool makes reality match it: reproducible, reviewable, and rebuildable from scratch. IaC is a category, not a specific product.

Terraform is the most widely used IaC tool. You declare resources in configuration files; it computes the difference between what exists and what you declared, then applies only the changes. Its signature trait is being cloud-agnostic: the same tool provisions multiple clouds via per-provider plugins. It keeps a state file tracking what it built so it knows what to change on the next run.

Docker is a packaging technology, distinct from provisioning. It bundles an app with everything it needs to run (code, libraries, system dependencies) into a container that runs identically on a laptop, a server, or the cloud, eliminating "works on my machine" problems. A container is like a shipping container for software: the outside is uniform so any host can carry it. Containers are lighter than virtual machines because they share the host's operating-system kernel instead of each booting their own.

Serverless is an execution model in which you write just a function and the provider runs it on demand, handling servers, scaling, and idle time. The name is a misnomer: there are servers, you just never manage them. Code runs when triggered, scales from zero to thousands automatically, and you pay only for execution time rather than idle capacity. The tradeoffs are less control and "cold starts," a brief delay when a function spins up after sitting idle.

An edge network is many small points of presence in cities worldwide, each close to end users, running lightweight code and cached data at whichever location is nearest the request. It minimizes latency by minimizing physical distance.

A hyperscaler datacenter is one of a few massive, centralized facilities in a handful of regions. Each offers maximum depth and power per location (every service, huge machines, full databases) but is physically far from most users. This model is best for heavy, stateful, centralized work.

The contrast is many-small-and-near versus few-big-and-far. By analogy, hyperscalers are a few enormous superstore warehouses: everything in one place, but you drive across the state. The edge is thousands of corner stores holding the common items a two-minute walk away. Most real systems use both: the edge for the fast front layer, a hyperscaler for the heavy back layer. The line between them is blurring as each encroaches on the other's territory.

These three layers relate cleanly: Docker packages the app, IaC and Terraform provision the infrastructure to run it, and serverless is one kind of infrastructure, one where the packaging and provisioning are largely abstracted away.