The terms that come up in every AI engagement, explained the way we'd explain them on a scoping call, not copied from a paper. Search or jump straight to a term.
A structured testing framework that measures an AI agent's performance against defined metrics (accuracy, correct tool use, cost) on a labeled dataset, used to catch regressions and validate improvements before deploying changes to production.
See an evaluation harness in production →A process where an AI agent autonomously chains together multiple steps, tool calls, or sub-agents to complete a task, as opposed to a single request-response interaction.
A system that uses a language model to decide which actions to take, in what order, to accomplish a goal, rather than following a fixed script. An agent can call tools, read their results, and change its next step based on what it learns.
The key difference from a simple chatbot is autonomy over a multi-step process, not just generating a response to one message.
Our LLM & Agent Applications work →A product or feature where the AI model is the core of what it does, with the architecture, data model, and UX designed around the model from day one, as opposed to AI bolted onto an existing product as an add-on feature.
AI-native vs. AI bolted on →A prompting technique where a model is asked to reason step by step before giving a final answer, which generally improves accuracy on tasks that require multi-step logic.
The cutoff value used to decide whether a model's output is trustworthy enough to act on automatically, or should be routed to a human for review. Setting this correctly is usually a business tradeoff between automation rate and error rate, not a purely technical decision.
See it in a real case study →The maximum amount of text (measured in tokens) a model can consider at once, including the prompt, any retrieved documents, and conversation history. A model can only reason about what fits inside this window.
When the real-world data a production AI system encounters gradually diverges from the data it was trained or evaluated on, which can silently degrade accuracy over time if not monitored for.
A numerical representation of text (or an image) that captures its meaning in a way that lets a computer measure how similar two pieces of content are. Embeddings are the foundation of semantic search and RAG.
Token & cost calculator →The process of further training an existing model on your own labeled data so it performs better on a specific, narrow task than prompting alone can achieve.
RAG vs. fine-tuning →The ability of a model to invoke a defined function or API with structured arguments, rather than just generating text, which is what lets an AI agent actually take action in external systems.
Our MCP & Agent Tooling work →Constraining a model's response to only use information from a specific, verified source (like retrieved documents or a database query result), rather than letting it generate from its general training. Grounding is the main defense against hallucination.
See grounding in production →When a model generates information that sounds plausible but is factually wrong or entirely made up, not grounded in any real source. This is the central risk any production AI system needs to be architected around, not just prompted away.
A system design where a human reviews or approves an AI system's output before it takes effect, typically for decisions with real cost or safety stakes, or when the system's confidence is below a defined threshold.
See human-in-the-loop design in practice →The process of running a trained model to produce an output, as opposed to training the model in the first place. Inference cost and latency are usually the main operational concerns once a system is live.
Estimate inference cost →A machine learning model trained on large amounts of text, capable of generating, understanding, and reasoning about language. Claude, GPT, and Gemini are examples of LLMs.
An open standard for connecting an AI model to external systems (databases, internal APIs, file storage) through a consistent protocol, so a tool built once can be used by any MCP-compatible AI client instead of writing custom integration code for every combination of model and system.
What is MCP? →The practices and infrastructure for deploying, monitoring, and maintaining AI systems in production: CI/CD for models and prompts, cost tracking, latency and drift monitoring, and infrastructure-as-code for reproducible environments.
See an MLOps engagement in production →An architecture where multiple specialized AI agents, each with a narrow domain and its own tools, are coordinated by an orchestrator agent, rather than one generalist agent trying to handle every part of a task.
See a multi-agent system in production →Running a model directly on a user's phone or device rather than sending data to a cloud server. Reduces latency and cost, and keeps data local, at the cost of using a smaller, less powerful model.
On-device vs. cloud AI →The logic that decides which agent, tool, or model to invoke at each step of a task, and how to combine their outputs. In a multi-agent system, the orchestrator is what routes work to the right specialist.
The practice of designing the instructions given to a model to reliably get the output you want, including structure, examples, and constraints. Distinct from fine-tuning, which changes the model itself rather than the instructions given to it.
An attack where untrusted input (a user message, a document the model reads, a piece of retrieved data) contains instructions designed to override the model's intended behavior. A real security risk for any AI system with access to tools or sensitive data.
See a real prompt injection audit →An architecture where a model's response is grounded in documents retrieved from a knowledge base at query time, rather than relying only on what the model learned during training. The standard approach for keeping an AI system's answers accurate and current.
RAG vs. fine-tuning →A model variant optimized to work through complex, multi-step problems internally before producing a final answer, generally at higher latency and cost than a standard model, used when accuracy on hard problems matters more than speed.
Search based on the meaning of a query rather than exact keyword matches, powered by embeddings. Finds relevant results even when the query uses different words than the source content.
See semantic search in production →The instructions given to a model before any user input, defining its role, constraints, and behavior for the entire conversation or task. Distinct from what the user types.
The basic unit of text a language model processes, roughly three to four characters of English text on average. Models are priced and rate-limited by token count, and context windows are measured in tokens, not words or characters.
Token & cost calculator →A database optimized for storing and searching embeddings, letting you find the most semantically similar documents to a query quickly at scale. Pinecone and Weaviate are common dedicated options, and pgvector adds the same capability to PostgreSQL.
Zero-shot means asking a model to perform a task with no examples in the prompt; few-shot means including a small number of examples to guide its output format and style. Few-shot generally improves consistency on structured tasks.
Ask us directly, or book a scoping call and we'll walk through it.