Agentic AI Costs Rise Even as Inference Prices Collapse

The paradox in the bill

The price of AI compute is falling fast, and many teams are finding that their AI spending keeps climbing anyway. Several analysts and vendors now point to the same cause: agentic workflows multiply the number of tokens consumed per task faster than hardware and model pricing can bring the per-token rate down.

Gartner states the tension most sharply. The firm predicts token costs will drop about 95% by 2030, yet inference costs for agentic workflows will rise more than fivefold over the next two years. 4 Its analysts say reasoning-capable agents can already cost up to 150 times more per task than a basic chatbot. They also describe a "massive inference tax" that builds as swarms of autonomous agents call one another before a user sees any result. 4

The enterprise data points the same way. A cost-statistics roundup citing the FinOps Foundation reports that 73% of organizations exceeded their AI cost projections in the past year. 3 The Foundation names three main causes: token volume that grew faster than expected as usage scaled, hidden token costs from reasoning models, and agentic chains that multiply consumption across steps. 3

Where the tokens go

Two mechanisms do most of the damage. The first is context inflation. One analysis contrasts a typical 2023 chat request of roughly 800 input tokens with a 2026 agentic step that routinely carries 50,000 tokens of memory, retrieved documents and tool-call history. 3 The second is looping. A workflow that chains 20 model calls uses roughly 20 times the tokens of a single request, and often neither engineering nor finance can see the breakdown by step. 3

TrueFoundry argues the real growth is worse than linear. In ReAct-style frameworks, each action's output is appended to the context before the next reasoning step. That pushes per-run token use toward O(n²) in the number of steps. 2 Continuous integration makes it worse still, because an automated PR-review agent can fire hundreds of times an hour on ordinary commit traffic. 2

The company's own example shows the scale. A CI/CD review agent averaged around 400,000 input tokens per pull request and produced an $8,400 first-month bill. 2 Gateway data traced $8,200 of that to a single pipeline. One step inside that pipeline accounted for 92% of its cost, and fixing it brought the monthly total under $800. 2 This is a vendor case study, but it shows a common pattern: most of the spend sat in one place, and nobody could see it until per-step data was available.

Cheaper hardware isn't the whole answer

The hardware side really is getting cheaper and more competitive. GPU cloud provider Spheron's April 2026 pricing puts an 8x H100 pod at about $1.90 per million tokens serving Llama 3.1 70B. By its own comparison, the same setup costs about $5.46 on AWS. 5 Its spot rates also undercut on-demand prices by a wide margin on newer chips, with the B200 dropping from $11.12 to $5.34 an hour. 5

Those same figures also show the limits of hardware savings. In Spheron's table, newer GPUs deliver more throughput but do not lower cost per token: an A100 configuration comes in around $1.67 per million tokens, against $3.18 for a B200. 5 Choosing the right GPU and buying spot capacity can trim a bill by a constant factor. Neither does anything about a workflow whose token count grows with every extra reasoning step or subagent. These numbers come from a provider promoting its own pricing, so treat them as directional.

Attribution as the real control

If the main lever is token volume, the main tool is visibility at the level of each agent run. Laminar's comparison of cost-tracking platforms makes per-run attribution its deciding criterion. 1 By its definition, that requires three things:

• every call logged with model, provider and a full token breakdown, including cached and reasoning tokens
• subagent calls rolled up into the parent run
• each run tagged by customer, feature, session and environment 1

It places tracing tools such as Laminar, Langfuse, LangSmith and Datadog at this layer, where costs are computed per span and summed into traces. 1 TrueFoundry makes a similar case for gateway-level breakdowns and budgets in CI/CD. 2

The takeaway

The sources disagree mostly on emphasis. Infrastructure vendors stress price per GPU-hour, while analysts and observability vendors stress volume. Taken together, the evidence favors the volume argument. Falling unit prices are real, but they set only the starting rate. The final bill depends on architecture: how much context each step carries, how many loops run, and how often agents are triggered.

Teams that treat agent cost as an observability and evaluation problem are likely to control it. That means measuring every run, alerting on outliers and pruning expensive steps. Teams that wait for cheaper GPUs may keep exceeding their budgets, as most organizations already have. 3