Ask most teams what an AI agent will cost to run and you'll get a number based on token pricing multiplied by expected usage. That number is almost always too low, often by a factor of two to twelve in our experience, because token cost is the easy, visible part of the bill. The expensive part is what happens around it.
Why the naive estimate is wrong
The naive calculation looks like: (input tokens + output tokens) × price per token × expected requests. That's a reasonable estimate for a single-turn chatbot answering one question per request. It badly underestimates an actual agentic system, because an agent rarely makes exactly one model call per user request.
What actually drives agent cost
- Tool calls and chained reasoning. A single user request to an agent can trigger multiple model calls: one to decide what to do, one or more to process tool results, one to formulate the final response. A task that looks like "one request" to the user might be four or five model calls under the hood.
- Retries and error handling. When a tool call fails, times out, or returns something the model doesn't expect, well-built agents retry or re-plan, which means more model calls for the same user action. Poorly instrumented systems can retry silently in loops that go unnoticed until the bill arrives.
- Context growth over a session. As a conversation or task gets longer, the context sent with every subsequent call grows too, since most agent architectures include prior turns and tool results in each new call. A long-running agentic session can cost far more per turn near the end than at the start.
- Retrieval overhead. If the agent uses RAG, every call that needs context also pays for embedding the query and including retrieved documents in the prompt, on top of the generation cost.
- Model tier mismatches. Using a large, expensive model for every step, including simple classification or routing decisions that a much cheaper model could handle reliably, is one of the most common and most fixable sources of inflated cost.
The estimation model we actually use
- Map the real task flow, not the ideal one. Walk through an actual representative task end to end: how many model calls does it realistically take, including tool calls, re-planning steps, and a reasonable retry rate based on tool reliability.
- Separate model tiers by task step. Not every step needs your most expensive model. Routing and simple classification steps often work fine on a smaller, cheaper model. Pricing each step at the model tier it actually needs, not a flat rate across the whole flow, usually cuts the estimate significantly.
- Model context growth explicitly. For multi-turn or long-running tasks, estimate token cost per turn as the conversation grows, not a flat per-turn average. Later turns in a long session can cost meaningfully more than early ones.
- Add a realistic retry and failure rate. Base it on the reliability of the tools and APIs the agent depends on, not on zero. Real systems have real failure rates, and retries cost real tokens.
- Price infrastructure separately from tokens. Vector database queries, embedding generation, logging and observability tooling, and hosting all have real costs that a pure token-price estimate leaves out entirely.
- Build in a buffer for usage patterns you haven't seen yet. Real users find edge cases and usage patterns a prototype's test set never covered. Budgeting for that discovery period, rather than assuming launch-day usage patterns hold forever, avoids an unpleasant surprise in month two.
What this looks like for a real engagement
During discovery, we build this cost model against the client's actual expected usage, not a generic assumption, before committing to a specific architecture. In more than one case, this has changed the technical approach entirely: switching a step from a large frontier model to a fine-tuned smaller model, restructuring a workflow to reduce redundant tool calls, or moving part of a mobile AI feature on-device specifically because the cloud-inference cost at real scale didn't work.
The goal isn't to minimize cost at all costs. It's to make sure the number you're planning around is the number you'll actually see once the system is handling real traffic, not the number from a spreadsheet that only accounted for the easy part of the bill.