OpenAI Jalapeño Chip Tops Nvidia Blackwell in Inference Benchmarks

A custom chip clears its first public test

OpenAI has published the first measured results for Jalapeño, the inference chip it designed and that Broadcom helped bring to production. On these benchmarks it does better than Nvidia's Blackwell-generation systems. In results released on August 25 across three open-weight models (GPT-OSS 120B, DeepSeek R1 and Kimi K2.5 1T), the company reported 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the systems it compared against. On highly interactive workloads the gap grew to between 2.1 and 4.1 times.3

The numbers were presented at Hot Chips, and the comparison targets were Nvidia's GB200 and GB300 rack systems running SemiAnalysis's public InferenceX benchmark.22 The timing was notable. The results landed just over a week after Nvidia agreed to backstop up to $105 billion in financing for OpenAI's data centers.22 Jensen Huang then had his answer on Nvidia's earnings call the next night: he described Jalapeño as a single-cloud accelerator while reporting $96.2 billion in quarterly revenue.25

My reading is that Jalapeño is a real engineering result, and the most credible challenge yet to Nvidia's dominance in inference. It is also a benchmark win under narrow conditions, from a chip that is not yet shipping. Both of those things are true, and the details of each matter.

From unveiling to benchmarks in two months

OpenAI and Broadcom unveiled Jalapeño on June 24. They called it OpenAI's first "Intelligence Processor" and the first accelerator in a compute platform planned to span several generations.1 At that stage OpenAI said only that early tests showed performance per watt "substantially better" than the current state of the art. It promised a detailed technical report later.1 Broadcom CEO Hock Tan put a number on it for Bloomberg, saying the chip was showing cost savings of roughly 50% compared with typical AI GPUs.15 Forbes said plainly that the figure came from Broadcom, was measured on workloads OpenAI picked, and had no independent check behind it.20

The August data partly fills that gap. SemiAnalysis said it ran InferenceX alongside OpenAI engineers in OpenAI's lab, and it called Jalapeño a generalized inference chip rather than one tuned narrowly for OpenAI's own models.4 The firm said the chip beat Blackwell on performance per watt in almost every scenario, and that its GSM8k accuracy scores matched those of Nvidia hardware.4 The Decoder gave a more careful account of who did what: OpenAI supplied the figures, and SemiAnalysis verified some runs on site.23

The hardware details help explain the result. SemiAnalysis reports that the chip runs at 700 watts on TSMC's N3P process. A B0 stepping now in the fab delivers 13.4 petaflops of MXFP4 compute, and HBM4 memory provides 15.4 TB/s of bandwidth per package.4 By the firm's measure, it has the highest HBM bandwidth per watt and the highest FLOPs per watt of any accelerator it compared.4 The architecture splits cores and memory into slices, so each core reaches its own share of HBM with low latency. This is meant to avoid the deep memory hierarchy that GPUs depend on.4 OpenAI's own explanation is similar: keep model state such as the KV cache in local memory, and cut data movement at every stage of inference.3

Where the coverage diverges: the Rubin question

The main point of disagreement is whether Jalapeño beats Nvidia's current platform or only the previous one. Tom's Hardware stressed that Vera Rubin was not in OpenAI's comparison, even though Rubin is the platform behind the first gigawatt of Nvidia systems OpenAI has agreed to deploy in the second half of 2026.22 IBTimes reported the same gap and noted that Rubin systems are already reaching customers.

SemiAnalysis took the opposite position. It argued Rubin is the right comparison because both chips use HBM4, and it found that Jalapeño's single-token throughput per megawatt beats the Rubin multi-token results Nvidia and CoreWeave published in July.21 The firm's CEO, Dylan Patel, said OpenAI was beating both Blackwell and Rubin, even though first-generation chips usually aren't competitive.23 Even so, SemiAnalysis found the two roughly tied on performance per total cost of ownership.21

XenoSpectrum gave the most useful corrective. The Rubin claim comes from SemiAnalysis placing two separately published datasets side by side, with different models, input lengths, inference methods and levels of software maturity. It was not a direct head-to-head test.27 The outlet also noted that Nvidia announced on August 24 that Vera Rubin NVL72 reached up to 30 times GB300's throughput per megawatt on its AgentX benchmark. That figure is also early, and SemiAnalysis has not reviewed it.27

The fair verdict is that Jalapeño clearly beats Blackwell and is roughly even with Rubin, with a possible edge. Tech Times reached the same conclusion.25 Calling it "better than Nvidia" without that qualification goes further than the evidence.

The fine print on the methodology

Three caveats deserve more attention than they got. First, the tests used a fixed workload of about 8,000 input tokens and 1,000 output tokens. Nvidia calls that a controlled single-turn setup and points to its agent-style AgentX benchmark as closer to real deployments.27 Second, the per-watt figures are divided by each chip's rated package power (700W for Jalapeño, 1,200W for GB200, 1,400W for GB300), not by measured facility power. Host CPUs, networking and cooling are left out.27 Jalapeño's sustained draw was reportedly under 550W, which makes the chip-level figures conservative but says nothing about whole-system efficiency.26

Third, the treatment of decoding techniques cuts both ways. Tom's Hardware noted that the main comparisons ran single-token prediction against GB300 configurations doing the same, although Nvidia deployments commonly use multi-token prediction in production.22 Against Rubin, though, the imbalance favors Nvidia, since Rubin's July results used speculative decoding and Jalapeño's did not.21 SemiAnalysis estimates speculative decoding cuts cost per token by about 3 to 5 times, so Jalapeño may still have room to improve.21 Some figures that circulated after the presentation, especially extreme hourly cost-of-ownership comparisons, are better treated as rough models than settled facts.

Custom silicon's moment, and the TPU comparison

Jalapeño adds OpenAI to a trend Google and Amazon started, building their own accelerators to reduce dependence on Nvidia GPUs.12 The Google comparison fits on technical grounds too: SemiAnalysis says Jalapeño's matrix engine uses a weight-stationary systolic array like the TPU's, but supports smaller shapes to avoid the performance cliffs that awkward matrix sizes can cause on large systolic arrays.4 Forbes offered a useful perspective: Google spent roughly a decade reaching its seventh TPU generation, Ironwood, its first built specifically for inference. Jalapeño is a first generation.20

Accounts of the development timeline differ, and the difference matters. OpenAI says the chip went from design to tape-out in nine months, helped by its own models, and calls it what it believes is the fastest ASIC development cycle on record.1 SemiAnalysis dates the start of design work to mid-2024 and puts the time from team hiring to tape-out at about 16 months.4 Both are fast. The nine-month figure, though, describes a narrower window than the headline suggests.

The software story may matter more over time. Tech Times reports that OpenAI programs Jalapeño with Gluon, its own kernel language built on Triton.25 SemiAnalysis observed that OpenAI models running on Nvidia GPUs helped design a chip that now threatens the CUDA moat.4

Why Nvidia isn't in trouble yet

Deployment, not benchmarks, will decide how much this matters. Small initial production runs come at the end of 2026, with bigger deployment in 2027 and a full ramp continuing into 2028. Until then, most of OpenAI's inference traffic will keep running on Nvidia hardware.25 Jalapeño also doesn't train models, which remains Nvidia's strongest ground.22 Manufacturing is a further constraint, since TSMC's advanced packaging capacity is reportedly sold out through 2026.20

The immediate effect is leverage. The New York Times noted that designing its own chips gives OpenAI more bargaining power with Nvidia and AMD.14 OpenAI describes Jalapeño as a first-party option alongside its partners' accelerators, not a replacement for them.7 Analyst Adrien Sanchez put it at the right level: this is a threat to Nvidia's inference margins, the fastest-growing part of the market, not a collapse of Nvidia's position.25 Broadcom may be the clearest winner. CNBC reports that OpenAI's partnership targets 10 gigawatts of custom chips.13 Forbes describes Broadcom as a toll booth that collects revenue from the custom chip programs of Google, Meta and OpenAI.20

Jalapeño shows that a large AI lab can build inference hardware on its first attempt that matches Nvidia's newest platform on efficiency. What it hasn't shown yet is that it can do so at gigawatt scale, on agent-style workloads, against Nvidia's software stack as it matures. That is the test for 2027.