OpenAI's Jalapeño Chip Just Beat Nvidia's Blackwell—But There's a Catch

At Hot Chips this week, OpenAI revealed benchmark data showing its custom inference processor outperforms Nvidia's top-tier chip on speed and power efficiency, but meaningful deployment won't arrive until 2027.

OpenAI just announced it built a chip that beats Nvidia. Yes, it's called Jalapeño—and no, that's not a marketing gimmick. This week at the Hot Chips conference, OpenAI published the first real benchmark data from this episode (video) and the full podcast coverage, showing actual test results on an independent benchmark called InferenceX, run by SemiAnalysis analysts. The results are substantial enough to reshape how we think about AI hardware dominance.

The Inference Problem OpenAI Is Solving

Training AI models is only half the battle. The harder problem—and the more expensive one—is serving those models at scale. Running millions of user requests through ChatGPT and OpenAI's API, fast and efficiently, is inference. It's slow. It's power-hungry. And it's currently bottlenecked by hardware designed before today's AI workloads existed. Every company running large language models is trying to solve the same problem: lower latency, higher throughput, less electricity wasted. But OpenAI has more riding on it than most. Instead of just buying more Nvidia GPUs, they built their own chip from scratch, in partnership with Broadcom.

What Jalapeño Actually Beats

On the InferenceX benchmark, Jalapeño outperformed Nvidia's Blackwell system—currently one of the market's most advanced AI chip platforms—on two critical metrics. Tokens per user measures response speed to individual requests. Throughput per kilowatt measures how much useful AI work you extract from every unit of electricity spent. Jalapeño won on both. According to Richard Ho, OpenAI's head of hardware, the chip delivers "a very, very significant performance advance over state of the art." He emphasized that Jalapeño achieves the inference holy grail: "more AI work per unit of power, while also returning responses more quickly." Normally you trade speed against efficiency. Jalapeño apparently does both.

How OpenAI Actually Built This

Here's where it gets genuinely surprising. OpenAI's own AI models assisted in developing the chip itself. AI helped design the hardware that will run AI. That's not hypothetical—that's what happened. The company also structured Jalapeño as a multigenerational platform, meaning future chips, models, memory systems, and networking are all being developed together in sync, rather than as separate pieces bolted on later. This full-stack approach is the real secret behind the performance gains. Because OpenAI controlled the entire stack, they could target specific bottlenecks in the inference process—particularly the prefill phase and communication phase, two stages that notoriously slow down response times. In OpenAI's words: "We designed Jalapeño to minimize data movement and communication delays." The KV cache and model state stay local, so the system spends less time shuffling data and more time generating answers.

The Timing Problem

But here's the catch nobody emphasizes enough: Jalapeño won't ship in serious volume for a while. Ho estimated "very small volumes" by the end of 2026, with meaningful large-scale deployment arriving in 2027. In the chip world, that's an eternity. By then, Nvidia's competition—and Nvidia itself—may have moved considerably forward. This isn't a "Nvidia is dead" moment. It's a "watch this space very closely" moment. OpenAI just proved it can actually compete in the chip race, not just talk about it. But the real test comes when Jalapeño actually scales.