OpenAI Jalapeño Chip Beats Nvidia Blackwell on Inference per Watt

What OpenAI reported

OpenAI has released the first benchmark data for Jalapeño, an inference chip it built with Broadcom. The company says the chip outperforms Nvidia's Blackwell-generation systems on two measures: how much AI work it does for each watt of power, and how quickly it returns answers. OpenAI presented the results at the Hot Chips conference on August 25, 2026. It tested three open-weight models: GPT-OSS 120B, DeepSeek R1 and the 1-trillion-parameter Kimi K2.5.4 Across those three models, OpenAI reported 1.5 to 1.9 times more work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the GB200 and GB300 rack systems it used for comparison.6

The tests used InferenceX, a public benchmark from the research firm SemiAnalysis, so the results do not rest only on internal slides.118 OpenAI also reported a bigger lead on interactive, chat-style workloads, with throughput 2.1 to 4.1 times higher than the comparison systems.46

The chip only runs trained models. It does not train them, and OpenAI says it will not sell it to anyone else.13 Limited deployments are planned for the end of 2026, with a larger rollout in 2027. A second-generation chip is already in development.1813

Efficiency was the goal from the start

The main story here is efficiency, not raw speed. Jalapeño is rated at 700 watts, compared with 1,200 watts for Nvidia's GB200 and 1,400 watts for the GB300. OpenAI scaled its results to each chip's published power rating. Reports from the Hot Chips presentation say the chip's measured sustained draw stayed at or below 550 watts on the tested workloads.101

On spec sheets alone, the chip looks weaker. One SemiAnalysis breakdown says Jalapeño has less than half the four-bit (FP4) compute of Nvidia's Vera Rubin and lower memory bandwidth, yet it delivers about twice the tokens per megawatt of a GB300 at a fixed speed of 100 tokens per second per user.15 The proposed explanation is a design modeled on Google's TPU, with deliberately small systolic arrays, the grids of arithmetic units that do the matrix math.15

Coverage of the architecture focuses on one point: the chip is built to move data as little as possible, because shuttling data between memory and processors is a major source of delay and wasted energy in serving language models.110 Running a model involves two phases. Processing the prompt (prefill) is limited by compute. Generating tokens one at a time (decode) is limited by memory bandwidth. Jalapeño reportedly uses different mixes of compute, memory and networking for each phase. It also keeps the KV cache, the stored context the model reuses for each new token, close to where it is needed.1 Networking is built into the design so that large models can run inside one connected system without waiting on links between chips.1

In my view, this is a lesson in hardware–software co-design, not a breakthrough in transistors. A chip built for a known set of workloads should beat a general-purpose accelerator on efficiency, and OpenAI's own finance chief describes the strategy in those terms.3 The surprise is how large the gap is for a first-generation product.

Fast tokens and cheap tokens at the same time

The result that matters most to people who study model efficiency is how the chip handles the tradeoff between throughput and latency. Accelerators are usually tuned either to serve many users cheaply, accepting slower responses, or to answer one user very quickly at the cost of efficiency. SemiAnalysis's Myron Xie said OpenAI's data showed Jalapeño ahead of Nvidia at every point on the curve that plots throughput per megawatt against per-user latency, not just at one chosen operating point.15 At the low-latency end, the chip reportedly reached about 700 tokens per second per user, compared with a competitive ceiling of about 350.15

OpenAI hardware chief Richard Ho made the same point to Bloomberg. He said most chips are good at either volume or responsiveness, while Jalapeño handles both. That would let OpenAI offer customers deployments tuned for lower cost or for faster answers.19 OpenAI also said the chip's lead grew on Kimi K2.5, the largest model it tested, and on unreleased internal frontier models. The company argues the design becomes more valuable as workloads get bigger.191

This matters most for agentic AI. When an agent chains many model calls in sequence, delays add up, so a chip that keeps latency low without giving up efficiency could make longer reasoning chains practical.12

AI helped design the chip

The second efficiency story is about how the chip was made. OpenAI President Greg Brockman said the chip was designed end to end in nine months, and he credited OpenAI's own models with part of the speed-up.2 SemiAnalysis's analysis says OpenAI's models cut about 8% from the area of the SIMD units and about 10% from the matrix engine during design. It also says AI-generated assembly kernels running to around 30,000 lines were used, code that even OpenAI's engineers reportedly cannot fully explain.15 Separate reporting says the B0 revision of the silicon improved performance per watt by as much as 25% over the first A0 revision.10

There is some disagreement over how much credit the AI deserves. One analysis says Ho stressed that a small, highly skilled team became more productive, not that models replaced engineers.10 SemiAnalysis's hosts went further, suggesting that model-driven design and programming may be OpenAI's most lasting advantage. They also noted the irony that Nvidia hardware trained the models now helping OpenAI move away from Nvidia.15 My reading is that the nine-month timeline is the more important claim. If it holds for later generations, OpenAI could revise its inference hardware about as fast as model architectures change.

Where the comparison breaks down

Coverage agrees on the raw numbers but not on how much they mean. The biggest caveat is memory. SemiAnalysis, which tested the chip in OpenAI's labs, called the Blackwell comparison "somewhat incomplete and unfair" because Jalapeño uses newer HBM4 memory while Blackwell uses HBM3e. The firm said Nvidia's HBM4-based Rubin platform is the fairer comparison.114 Bloomberg reported that Jalapeño was not tested against Vera Rubin, which had just begun shipping.19

Accounts of the chip's position against Rubin differ. One summary of SemiAnalysis's podcast says Jalapeño beats Vera Rubin on DeepSeek, Kimi K2.5 and GPT-OSS.15 Forbes reported that SemiAnalysis found the two producing nearly the same output tokens per dollar. Forbes added that Rubin's numbers relied on speculative decoding, a technique that drafts several tokens ahead and then checks them, while Jalapeño's did not. Taken together, the fairest conclusion is that Jalapeño clearly beats Blackwell and roughly matches Rubin, an impressive result for a first chip but not a clean win over Nvidia's newest hardware.

There are other caveats. Chip power ratings understate what a data center actually draws: a 128-chip rack pulls about 130 kilowatts, and the full two-rack system approaches 160. Rubin is shipping, while Jalapeño was still at the engineering-sample stage. One analysis notes that no outside lab had published an independent re-run of OpenAI's exact head-to-head figures.2 Reports also disagree on how specialized the chip is. Some describe it as tuned closely to OpenAI's own models.6 According to Tom's Hardware, as summarized elsewhere, Ho rejected that idea and called it a general-purpose inference engine.8 The strong results on DeepSeek and Kimi support Ho's view.

Why power matters more than price

The strategic case for the chip rests on power, not cost alone. Forbes cited International Energy Agency figures showing electricity demand from AI-focused data centers rose 50% in 2025, and noted that grid connections are slow enough that some developers are building their own power generation. When power is the scarce resource, work per watt determines how much AI capacity a company can actually run. Analysts made the same point to CNBC: savings on power, cooling and electrical distribution add up across a large deployment.11

The effect on Nvidia's business is real but limited. Yole Group's Adrien Sanchez called Jalapeño a threat to Nvidia's inference margins, the fastest-growing part of its market, while acknowledging that Nvidia still controls most AI compute and benefits from its CUDA software lock-in.11 SemiAnalysis's cost model makes the margin argument directly: OpenAI pays Broadcom's design fees instead of Nvidia's full markup.15

OpenAI's own executives play down any break with Nvidia. Ho told Axios the company needs compute from Jalapeño, Cerebras, Nvidia and AMD.13 OpenAI's CFO Sarah Friar described Nvidia as the platform for training, and called AMD's MI455 "a very viable alternative in many cases."3 Jalapeño does not end OpenAI's reliance on Nvidia. It is OpenAI's first evidence that it can make inference cheaper by redesigning the hardware, not just by making models smaller. The real test is whether those gains hold up in production at scale during 2027.