Nvidia Vera Rubin Reaches Google Cloud as TPU Prices Pressure Costs

Nvidia's Rubin platform reaches cloud customers

Nvidia's newest data-center platform is now running for paying cloud customers. At GTC Berlin on October 21, Nvidia and Google Cloud said the first A5X instances built on Vera Rubin NVL72 racks are live with an initial customer. They plan general availability in November 2026 and a later expansion into Europe.25 The companies also repeated an earlier plan to scale Vera Rubin capacity to as many as 960,000 Rubin GPUs across several data-center sites.23 Google first described A5X at Cloud Next in April. At that event, Nvidia said the instances could cut inference cost per token by up to 10x and raise token throughput per megawatt by a similar factor compared with the previous generation.25

Read on its own, this is a routine product milestone. But it arrives while Google is also accelerating its own Tensor Processing Units, publishing hard prices for them, and spreading its custom-silicon work across several chip designers. That makes the announcement useful for understanding where AI compute is heading. The best reading is not "Nvidia versus custom chips." The two are converging, and the contest is now over the cost of a token rather than peak chip performance.

What Rubin actually is, and where accounts differ

The core specifications are consistent across coverage. A Rubin GPU has 336 billion transistors, 288GB of HBM4 memory and 50 petaflops of FP4 inference performance.21 The flagship NVL72 rack pairs 72 Rubin GPUs with 36 Vera CPUs. It is rated at 3,600 petaflops of NVFP4 inference, with 20.7TB of HBM4 and full liquid cooling.24 Memory bandwidth per GPU is about 22TB/s, up from 8TB/s on Blackwell. Power use roughly doubles, to an estimated 2,300 watts per GPU.21

The timeline is less tidy. Nvidia said in January that Rubin was in full production, with partner products due in the second half of 2026.26 At its March GTC event, it said seven Rubin-platform chips were in full production.26 A separate account dates the formal "ramping into full production" announcement to May 31, with broad customer shipments starting this fall.22 Even the chip count varies: some coverage describes a six-chip baseline and treats the seventh as optional.21 One retrospective notes that trackers place the full-production milestone anywhere from the first quarter to June 1. All of them agree on the overall pattern, though: qualification in late 2025, a ramp through the first half of 2026, and deliveries to major clouds by midsummer.26 Korean outlet ChosunBiz reported in late July that the first NVL72 racks had reached Google Cloud, Microsoft Azure, Oracle Cloud and CoreWeave.26

The difference matters less for buyers than for anyone judging Nvidia's execution. "Full production" has become a flexible term in Nvidia's messaging. The milestone that counts, systems running for customers, appears to have arrived on schedule.

The roadmap has also changed in ways that show where Nvidia sees demand. Rubin CPX is a GDDR7-based chip that Nvidia announced in September 2025 to handle the context-processing (prefill) stage of long-context inference. It has not shipped, and Nvidia has given no current date for it.29 Nvidia left CPX out of the GTC 2026 keynote and showed the Groq 3 LPX rack for low-latency decoding instead. Tom's Hardware reported that Nvidia executive Ian Buck said CPX was pulled so the company could focus on LPU-based decoding this year.29 Separately, Rubin Ultra was reportedly cut back from four dies to two because of packaging yield problems at TSMC. Nvidia still claims a 3.5x improvement in inference throughput per watt over Blackwell B300.21 Both changes point the same way: Nvidia is tuning its products for inference economics, not only for headline training performance.

Google's dual role: Nvidia's customer and its most credible rival

Google sits at the center of this story because it buys Nvidia hardware and sells a competing product at the same time. In April, it split its eighth-generation TPU into two chips for the first time: TPU 8t for training and TPU 8i for inference.13 According to Google's description, the 8i carries 288GB of high-bandwidth memory and 384MB of on-chip memory.13 A TPU 8t superpod scales to 9,600 chips and 121 exaflops at FP4.15

The supply chain behind these chips has become a story of its own. Broadcom says production shipments of TPU v8i began in its fiscal third quarter and should reach high volume in the fourth.11 Broadcom also has a five-year agreement with Google covering four TPU generations through version 11.11 In July, Google signed a deal with Marvell for accelerators, storage and network controllers, and near-memory compute tied to the TPU ecosystem.11 Under that deal, Google holds a warrant for roughly 59 million Marvell shares. The warrant vests as Google's purchases accumulate, with a ceiling of $120 billion in cumulative revenue.11 Other coverage puts Google's potential investment at about $12.2 billion, for roughly 6.7% of Marvell.20 Analyst Ming-Chi Kuo has reported that MediaTek is the exclusive development partner for an upgraded TPU v9 variant.19

One investor-focused report framed the Marvell deal as a threat to Broadcom's AI revenue.16 The more careful reporting describes it differently: Marvell is adding to Broadcom's role rather than replacing it, and JPMorgan calls Broadcom the "primary supplier" for the term of its agreement.11 That second reading is more convincing. Google is building a multi-vendor arrangement so that no single partner, whether Nvidia, Broadcom or anyone else, can set its prices.

The external customers are now large. In April, Anthropic agreed to multiple gigawatts of next-generation TPU capacity starting in 2027, which a Broadcom filing put at about 3.5 gigawatts.13 Meta has reportedly signed a lease for Google TPU clusters worth tens of billions of dollars.20 Google's finance chief has acknowledged that TPU revenue is a small share of the total in 2026, with most orders to be filled in 2027.20 For now, then, the TPU threat to Nvidia shows up mainly in contracts. It will show up in shipped volume later.

Inference costs: list prices versus negotiated deals

Cost is where the competition is clearest. In September, Google published a list price for its seventh-generation Ironwood TPU: $12.00 per chip-hour on demand in Iowa, $6.00 with flexible scheduling, and $5.40 on a three-year commitment.18 SemiAnalysis estimated that Anthropic's negotiated rate is about $1.60 per TPU-hour, a small fraction of the list price.18 By comparison, Nvidia's B200 rents for about $9.36 per hour on demand on marketplaces, and H100s average roughly $2.69 to $3.00.18

The gap between list and negotiated prices is the key point. The cost of AI compute now depends mostly on who you are. Frontier labs with gigawatt-scale commitments pay far less than the advertised rates. Everyone else pays on-demand prices or rents from smaller GPU clouds.

On those smaller clouds, token prices are falling. One provider's figures show H100s at $2.64 per hour on demand and B200s at $9.95.5 It estimates that a 70-billion-parameter model run in FP4 on spot-priced B200s costs about $0.047 per million tokens, against roughly $0.227 for an H100 at on-demand rates in FP8.5 Another analysis estimates that B200 systems cut output-token costs by about 59% compared with the H100. It also finds that custom chips such as AWS Trainium2 and Google TPU v5p deliver under $0.35 per million tokens on reserved pricing.4 These figures come from vendors and benchmarks, and they depend on assumptions about batch sizes and utilization. Still, the direction is consistent across them: every new hardware generation and lower-precision format pushes the cost of a token sharply down.

Other hyperscalers are making similar claims. Microsoft launched its Maia 200 inference accelerator in January and says it delivers 30% better performance per dollar than its existing systems.2

The memory bottleneck

The main limit on falling costs is memory, not processing power. One manufacturing-cost analysis estimates that HBM makes up 30% to 64% of an accelerator's build cost. It also estimates that HBM alone costs about $6,500 per GB200 superchip, roughly twice the cost of building an entire H100.10 The same analysis puts Nvidia's gross margins on its data-center parts at an estimated 78% to 88%.10 That margin is exactly what Google's TPU business, with its vertical integration and multiple suppliers, is designed to avoid paying.

Memory demand is also spreading beyond data centers. AI facilities are reported to be taking about 70% of global memory output, and LPDDR5X contract prices rose 89% in the second quarter. Intel's CPU prices have climbed a cumulative 30% over three increases.9 Huang confirmed in June that SK Hynix, Samsung and Micron are all qualified to supply HBM4 for Rubin.22 Even so, supply-chain guidance warns that the largest cloud providers' orders are first in line for memory.28

What it means

The overall picture is clear. Nvidia has kept its lead: estimates give Google's TPU about 8% of the 2026 AI chip market, and AMD 6% to 8%.13 Rubin is shipping to every major cloud, including Google's. But the main argument for Rubin is now about cost: lower cost per token and more throughput per megawatt. That is the same argument Google, Amazon and Microsoft make for their own chips.

When the market leader and its largest customers compete on the same measure, pricing pressure follows. The labs that can commit gigawatts of capacity are already capturing most of the savings. For everyone else, the cost of AI inference will keep falling, but it will be shaped as much by HBM supply and contract terms as by any new chip.