AI Inference Costs to Drop 90% by 2030 as Nvidia Rubin Looms

A forecast with a built-in catch

The cost of running large language models is about to fall sharply, according to Gartner. The same forecast says that most companies paying for AI should not expect a smaller bill.

In a forecast released March 25, 2026, Gartner said that by 2030, running inference on an LLM with one trillion parameters will cost generative AI providers more than 90% less than it did in 2025.12 The firm also projects that 2030-era LLMs will be up to 100 times more cost-efficient than the earliest models of similar size built in 2022.12 Will Sommer, the Gartner senior director analyst behind the work, listed the drivers: more efficient semiconductors and infrastructure, new model designs, higher chip utilization, more use of inference-specialized silicon, and running some workloads on edge devices.12

The catch sits inside the same release. Gartner says provider savings will not be fully passed on to enterprise customers. It also says frontier capabilities will need far more tokens than today's mainstream applications. Agentic models, for example, use 5 to 30 times more tokens per task than a standard chatbot.16 Because token consumption is expected to grow faster than token costs fall, Gartner expects overall inference spending to rise.12 Sommer was blunt about it, telling CIO Dive that "the customer isn't going to see all of this money."13

The forecast is really two claims, and they have to be read together. Silicon is getting dramatically cheaper per unit of work. That cheapness is mostly being spent on more ambitious workloads, not banked as savings.

The hardware side: Nvidia's cost-per-token arms race

Gartner's forecast fits closely with what chipmakers have been promising all year. Nvidia has made cost per token its main marketing metric. Its inference platform page calls cost per token "the only metric that matters" for total cost of ownership.4

The generational numbers Nvidia publishes are large. The company says its Blackwell Ultra GB300 NVL72 rack delivers about 35 times lower cost per million tokens than a Hopper-based HGX H200 system. Its figures are $0.12 against $4.20, even though the Blackwell system's hourly cost per GPU is roughly double.4 Nvidia cites SemiAnalysis InferenceX benchmarks putting GB300 NVL72 at $0.123 per million tokens at 116 tokens per second per user as of April 2026.6

Nvidia says the next step is bigger still. At CES in January, it introduced the Vera Rubin NVL72 platform and claimed up to five times Blackwell's inference performance and as much as a tenfold cut in cost per token for mixture-of-experts inference. Cloud availability is promised for the second half of 2026.8 Each Rubin GPU is rated at 50 petaflops of NVFP4 inference, and Nvidia says training MoE models takes a quarter as many GPUs as on Blackwell.8 Later coverage of GTC repeated the one-tenth token-cost claim for agentic AI, advanced reasoning and large-scale MoE inference. It also put estimated NVL72 rack prices at roughly $3.5 million to $4 million, though Nvidia has not disclosed pricing.10

These claims should be read carefully. They are vendor figures, often measured on specific models under ideal conditions. The headline "two cents per million tokens" on B200, for example, comes from GPT-OSS-120B.3 Still, the direction lines up with Gartner's frontier scenarios, which model costs on cutting-edge chips. Gartner's "legacy blend" scenarios, based on a mix of older semiconductors, show considerably higher costs.16 So hardware strategy is not a side issue in the forecast. It is the variable that separates the optimistic curve from the pessimistic one.

Software may matter as much as silicon

Across the coverage, much of the cost decline comes from software running on the same chips, not from new chips. Nvidia says B200 cost per million tokens on GPT-OSS-120B dropped from $0.11 at launch to $0.02 within two months, a fivefold gain from software alone.4 An analysis of InferenceX data found an even bigger swing. On a GB300 rack serving DeepSeek R1, turning on multi-token prediction cut costs from $2.35 to $0.11 per million tokens, about 21 times.2

That supports Gartner's emphasis on chip utilization and model design. Inference provider Inworld AI notes that a GPU-hour costs the same whether it produces 2,000 or 27,000 tokens per second, so utilization drives the unit price.5 Public B200 list prices also varied widely, from $3.49 to $14.24 per GPU-hour across clouds in April 2026.5 On this evidence, a company's cost per token depends as much on how it serves models and where it rents capacity as on which chip it buys.

Inference-specialized silicon is no longer a side show

Gartner's mention of "inference-specialized silicon" points to the most competitive part of the chip market. Here the reporting diverges more.

AMD has a real case. In one SemiAnalysis benchmark, its Instinct MI355X served GLM-5 at $0.22 per million tokens against $0.30 for Nvidia's B200 in the same configuration, roughly 27% cheaper on that workload.2 AMD also said the MI355X landed within 92–104% of Nvidia's B300 in MLPerf serving scenarios. But MLPerf measures throughput, not dollars per token.2

Google is pressing from a different direction. It says its inference-focused TPU 8i improves performance per dollar by up to 80% over the earlier Ironwood TPU, especially for large MoE models at low latency.24 One investment analysis argues that serving Gemini only on TPUs is a key reason Google's cost per token is lower. It points to Anthropic's large Google Cloud commitments as outside validation.24

The Groq story shows how Nvidia is responding. In April, a Nebius expert quoted by Wccftech put Groq's chips at 5 to 10 cents per million tokens, against 25 cents for Nvidia's Blackwell parts, and said they were nearly twice as fast. The competitive framing changed after Nvidia licensed Groq's inference technology in a reported $20 billion deal in December 2025. The Groq 3 LPU now ships inside the Vera Rubin platform.25 Nvidia has absorbed a leading inference architecture rather than fight it. That suggests the company treats specialized inference silicon as a threat serious enough to own.

Estimates of how far alternatives have actually displaced Nvidia vary. One industry overview still puts Nvidia at about 80% of the AI accelerator market and expects that share to erode slowly rather than collapse.27 More aggressive commentary predicting Nvidia's decline is better treated as opinion than as settled analysis.28

Where the numbers disagree

The sources mostly agree on the shape of the story, but the numbers drift between outlets. Gartner's March release says more than 90%.12 Computerworld's August coverage of a follow-up report says token costs will fall 95% by 2030.18 Retrospective figures vary even more. One analysis says GPT-4-class inference fell about 1,000-fold, from $20 to $0.40 per million tokens, between late 2022 and early 2026.9 Another uses a 280-fold figure for per-token price declines since 2022.13 These differences come from different baselines and capability tiers, but together they show a cost curve that is unusually steep by any measure.

Gartner's August follow-up does the most to clarify the forecast. The firm now predicts that provider-side inference costs per agentic workflow will rise more than fivefold through 2028.20 Its Tokenomics Model found that planning and learning tasks cost providers 8 to 10 times more per token than basic workflows.18 Sommer and co-author Sabine Zimmerhansl say the market is "captured by a token-deflation illusion."18 As one analysis points out, the 90% decline and the fivefold increase measure different things, so they do not contradict each other.20

What it means

My reading: Gartner's forecast is less a prediction of cheap AI than a description of where the value will go. Falling hardware costs will mostly benefit providers and chipmakers that keep their fleets on the frontier curve. Gartner's own split between frontier and legacy scenarios implies that operators stuck on older hardware will be at a structural cost disadvantage.16

For Nvidia, that is a good position. Every claimed tenfold improvement gives customers a reason to upgrade instead of waiting, and the agentic demand Gartner describes keeps the volume of tokens rising. For AMD, Google and specialized inference chipmakers, the shift to cost-per-token pricing gives them a fairer basis for comparison than hourly GPU rates did.

For enterprises, Gartner's advice is consistent across both reports. Send routine, high-volume work to small and domain-specific models, keep frontier models for complex, high-margin reasoning, and treat orchestration as the main way to differentiate.1618 Over the next four years, cheaper tokens are likely to show up as more ambitious AI products rather than lower bills.