So OpenAI Built a Faster Chip Than Nvidia. Now Ask Who's Holding the Leash.

So OpenAI Built a Faster Chip Than Nvidia. Now Ask Who’s Holding the Leash.

Let me tell you something, straight. Last Tuesday, OpenAI walked into Hot Chips and dropped a bomb. Their custom inference chip, Jalapeño, co-built with Broadcom, just beat Nvidia’s flagship GB300 on the only metric that matters when you’re paying the electric bill: throughput per kilowatt. We’re talking 1.5x to 1.9x better efficiency, and 1.7x to 3.6x lower latency on the SemiAnalysis InferenceX suite. That is not a rounding error. That is a statement.

But here is the part that should make every founder sitting on a hardware budget stop and think. One week before they published those numbers, Nvidia agreed to hand OpenAI up to $105 billion in financing for a data center campus in Ohio. Let that sink in. OpenAI just proved they can build a chip that embarrasses Nvidia’s current generation, and then they turned around and borrowed a hundred billion dollars from the guy they’re trying to dethrone. That is not a power move. That is a hostage situation wearing a business suit.

The Jalapeño Spec Sheet: What They Actually Built

Let’s give credit where it’s due. The hardware is real. Jalapeño is a 700W part that runs at or below 550W sustained, packing 216 GiB of HBM4 memory with 15.4 TB/s of bandwidth. Compare that to the GB300’s 288GB of HBM3E at 1,400W. Do the math yourself—that is roughly 50% more memory per watt of rated power. They built this thing on a TSMC 3nm-class process, and they went from RTL to tapeout in nine months. Nine months. That is a startup pace, not a semiconductor giant pace.

The benchmarks covered three open models: GPT-OSS 120B, DeepSeek R1 670B, and Moonshot AI’s 1-trillion-parameter Kimi K2.5. At low-latency operating points, OpenAI claims an absurd 8.6x to 104.3x more throughput per kilowatt. SemiAnalysis, who ran the tests with OpenAI engineers in OpenAI’s own lab, said it is “beating every Nvidia, AMD, and Google chip we have been able to test.” Bold words. And the deployment plan is real—OpenAI starts putting Jalapeño in its own data centers later this year. They have a 10GW agreement with Broadcom signed last October. A second-gen chip is approaching tapeout within months. Concept work on gen three is already underway.

So yes. The chip is real. The engineering is impressive. But if you are a founder reading this and thinking “Nvidia is finished,” pump the brakes. Because I read the fine print, and the fine print has teeth.

The Reality Check: What the Benchmark Didn’t Tell You

First, Jalapeño does not train models. It cannot train models. It will never train models. Training is where Nvidia remains completely unchallenged, and that is not an accident—that is the moat. CUDA is not just a software stack; it is a decade of institutional knowledge, debugging, and optimization that no custom ASIC is going to replicate overnight. You do not beat a moat by building a faster boat. You beat it by draining the water, and OpenAI has not even started digging.

Second, they did not test against Vera Rubin. That is Nvidia’s next-generation platform, the one slated to power the first gigawatt of Nvidia systems OpenAI agreed to deploy in H2 2026. So the benchmark is real, but it is also a snapshot of the past. The machine that is actually coming to the fight was not in the ring.

Third, the numbers are normalized to published package TDP. That is a clean way to measure silicon efficiency, but it is not the way you measure your electric bill. The appendix using all-in utility power per accelerator—1.18kW for Jalapeño versus 2.55kW for the GB300—produces much narrower gaps. And when you run the GB300 with multi-token prediction, which Nvidia deployments commonly use in production, the peak efficiency lead shrinks to roughly 1.5x. Still impressive. Not the blowout the headline suggests.

Fourth, SemiAnalysis ran the tests in OpenAI’s lab with OpenAI engineers present. I am not saying the numbers are cooked. I am saying that when the vendor is in the room, the test is a collaboration, not an audit.

Two Readings: The Liberation Story and the Business Reality

Here is where I ask you to hold two thoughts in your head at the same time, because both are true.

Reading One: The Liberation Story. OpenAI finally has its own inference silicon. The efficiency gains are real. Nvidia’s pricing power on inference is under direct threat for the first time in years. If you are running large-scale inference workloads, the prospect of a credible alternative to Nvidia is not just good news—it is leverage. You can negotiate. You can threaten to walk. That is a genuine shift in the balance of power.

Reading Two: The Business Reality. This chip cannot train. It runs through TSMC and HBM queues that Nvidia dominates. And one week before publishing these benchmarks, OpenAI accepted $105 billion in financing from Nvidia. That is not independence. That is a golden cage. OpenAI’s VP of hardware, Richard Ho, told Bloomberg: “Nvidia is a really good partner, and we continue to need a lot of Nvidia.” Read that quote again. They are not escaping. They are diversifying their dependency.

Both readings are true. The chip is a genuine breakthrough. The business relationship is a genuine trap. And the trap is not just financial—it is physical.

The Real Bottleneck: HBM Is the New Oil, and Nvidia Owns the Refinery

Here is the part that keeps me up at night, and it should keep you up too. HBM is the tightest commodity in semiconductors right now. Samsung, SK hynix, and Micron have sold their HBM capacity through 2027. SK hynix’s CEO warned that 2027 will be the worst year of the crunch. Micron told the same Hot Chips conference that HBM consumes roughly three times the wafer area of DDR5. That is not a supply chain. That is a bottleneck with a padlock.

Now watch what happens. OpenAI is scaling Jalapeño across a 10GW Broadcom agreement. That makes them a substantial new claimant to HBM4 supply. But Nvidia currently dominates that supply through multi-year allocation deals with SK hynix. So OpenAI’s independence play runs straight through the same memory queues Nvidia controls. You can design the best chip in the world, but if you cannot get the memory to feed it, you have built a very expensive paperweight.

This is the cross-pollination that most analysts miss. The “end of Nvidia” narrative is not about silicon design. It is about memory allocation. And on that front, Nvidia is not losing. They are getting paid to rent the cage while the bird learns to fly.

What This Means for Independent Hosting Providers

Now let’s talk about you. Because you are not OpenAI. You are not building a 10GW data center. You are trying to keep your margins alive while the giants fight over the table scraps.

First: Watch what happens when OpenAI’s deployment actually scales and takes HBM allocation. That is not a hypothetical. That is a demand shock coming to a supply chain near you. When OpenAI starts pulling HBM4 in volume, secondary GPU and inference pricing will move. If you are running inference workloads, your cost basis is about to get volatile. Plan for it.

Second: Do not build capacity bets on a single-vendor narrative. The Jalapeño benchmark is real, but it is one data point. Nvidia is not dead. AMD is not dead. Google is not dead. The market is going to fragment, and fragmentation is your friend—if you stay flexible. Do not lock your entire infrastructure into one vendor’s roadmap. Keep your options open, because the only certainty is that the landscape will shift again.

Third: The inference economics are shifting. Per-watt inference is where the margin fight happens next. If you are not already measuring your cost per inference, not just your cost per GPU, you are flying blind. The companies that win the next five years are the ones that optimize for efficiency, not just raw capacity. Start measuring. Start optimizing. Start treating power as your primary constraint, not your afterthought.

Fourth: Lock your own hardware orders now. HBM constraints are going to keep biting every tier of the supply chain, and that means GPU availability is going to stay tight. If you need hardware in the next 18 months, do not wait. Sign the contracts. Pay the deposits. The worst position to be in is needing capacity during a shortage, because that is when the pricing power belongs entirely to the seller.

The Truth Bomb

Here is the bottom line. The chip is real. The escape is not complete. OpenAI built a faster inference engine, but they are still renting the road from the guy they are trying to overtake. Nvidia is not losing this game. They are getting paid $105 billion to finance the cage while the bird learns to fly. And when the bird finally takes off, it will still need to buy its food from the same supplier.

That is not a victory lap. That is a warning. The hardware revolution is real, but the supply chain is the moat, and Nvidia still owns the bridge. Plan accordingly.

— Allan Ali, Founder

This article was produced with AI-assisted research and editorial support. Sources: Bloomberg, Tom's Hardware, DatacenterDynamics, OpenAI, SemiAnalysis.