Nvidia Put a Rival's Name on Its Flagship Chip — and the AI Buildout Just Changed Direction

Nvidia Put a Rival's Name on Its Flagship Chip — and the AI Buildout Just Changed Direction

I've been running hosting infrastructure for over a decade, and I've learned to read the tea leaves when the big boys make announcements. So when Nvidia woke up this morning and announced that its new inference chip — the thing it's betting the next phase of its empire on — is called the Nvidia Groq 3 LPX, I stopped scrolling.

Let me say that again slowly. The world's most valuable chip company, the one that sells the overwhelming majority of AI accelerators on the planet, just shipped a flagship product with its biggest rival's name stamped on it. That's not a partnership announcement. That's a confession with a press release attached.

And it's happening two days before Nvidia reports earnings on Wednesday. Nothing Nvidia does this week is an accident.

The Chip With the Rival's Name on It

Here's what actually got announced today, straight from the NVIDIA Newsroom: Nvidia Groq 3 LPX is in full production. It's a rack-scale, low-latency inference accelerator built to sit alongside the Vera Rubin NVL72 — the liquid-cooled AI factory platform. In an Artificial Analysis benchmark running Gemma 4 31B, an open agentic model, it hit 3,400 output tokens per second on 100,000-token long-context workloads — 4x faster than the nearest alternative platform, per Nvidia.

Here's the part that should make every independent hosting provider sit up. The chip is based on intellectual property from Groq — the LPU architecture — under a $20 billion licensing deal signed late last year. Each LPX rack carries 256 LPUs with 128GB of on-chip SRAM, 150 TB/s of bandwidth, and direct chip-to-chip links into NVL72. Rubin GPUs handle the prefill — chewing through the prompt — while the LPUs handle the latency-sensitive decode, the part where tokens actually come out one at a time.

And here's the kicker: this wasn't the first plan. Nvidia originally designed its own chip for this job — the Rubin CPX, a purpose-built long-context inference GPU announced back in September 2025. It got cancelled at GTC 2026 and replaced by the Groq design. Nvidia spent $20 billion to buy what it couldn't build fast enough itself.

The Inference Turn — Where the Money Actually Moves

This isn't a product launch. It's a pivot. For three years the entire AI buildout has been a training story: buy more GPUs, train a bigger model, raise more money, repeat. Training is a finite problem — you train the model, you're done, you move to the next one. Inference is the opposite. Every prompt, every tool call, every reasoning step, every agent loop burns tokens forever, 24/7/365. That's the "token factory" language Nvidia is using now — turning ever-growing volumes of tokens into revenue.

The analysts quoted in the coverage put it better than I can. Sanchit Vir Gogia at Greyhound Research said training rewards "massive parallelism and brute-force scale," but inference — especially long-context and agentic — is sensitive to latency, memory movement, cache behavior, and cost per delivered token. Matt Kimball at Moor Insights & Strategy summed up the whole shift in one line: training is "forget budget, forget power, let's get this model trained ASAP," while inference is persistent, sustained performance that has to happen everywhere, all the time, for every user.

Nvidia is telling you where the next decade of compute revenue comes from. It's not the training run. It's the toll booth on every token.

The Bottleneck Nobody's Watching — Decode Memory

Here's the pattern I've seen a dozen times in this buildout: the obvious bottleneck gets all the headlines while the real constraint hides underneath. The headline was "Nvidia's new inference chip is 4x faster." The real story is why a GPU company had to license an SRAM chip at all.

The constraint is the KV cache — the memory that holds the context while a model generates. For a Llama 3 70B-class model, every single token of context costs roughly 320KB of memory. A million-token context window means a third of a gigabyte of cache per sequence, just sitting there while the model thinks. GPUs with HBM are phenomenal at raw compute, but decode is a memory-bandwidth problem, not a FLOPS problem. SRAM — the LPU's entire bet — is what actually feeds the token generator fast enough. That's why Nvidia claims up to 35x higher inference throughput per megawatt on these systems. That's not marketing fluff; that's the physical difference between memory architectures.

For anyone running real infrastructure: the era where "just add GPUs" solved everything is over. The next constraint is memory bandwidth and interconnect, and it's a wall, not a slope.

The Full-Factory Squeeze

And don't miss the rest of today's announcement, because it tells you Nvidia is no longer selling chips — it's selling the entire factory. Alongside the LPX, Nvidia rolled out:

Spectrum-X Multiplane — a networking architecture that splits each server's connection into parallel "planes" and scales to 512,000 GPUs without adding a third network tier. CoreWeave has already deployed it in production.

Scale-In — the fifth pillar of Nvidia's networking, powered by BlueField-4 and DOCA, accelerating security, storage, and operations inside the AI factory.

NVLink Fusion — which lets hyperscalers plug their own custom XPUs and CPUs into Nvidia's scale-up fabric, "decoupling data center buildout from silicon readiness."

Read that last one again. Nvidia is so confident in its lock-in that it's now letting customers bring their own silicon — as long as it plugs into Nvidia's rails. That's the move of a company that knows the moat isn't the chip anymore. It's the network, the software, the rack, the whole ecosystem.

And Now They're Putting It in Space

Then there's the part that made me shake my head, because it's the most honest thing this industry has said all year: SpaceXAI announced it's building its future AI architecture around Vera Rubin "from data centers on Earth to orbital satellites." The first Starmind AI satellite is planned around a Vera Rubin NVL72 system optimized for space operations — 30-meter-tall satellites with 75-meter solar-array wingspans.

There are two ways to read this. Reading one: ambition. Orbital inference for global low-latency coverage, compute where the sun never sets. Reading two — and I've been around long enough to know which one is real — is that the AI buildout has hit the physical wall on Earth. Power queues measured in years. Water fights. Community consent collapsing. So the industry's answer to "we can't find power, land, and cooling on the planet" is literally "then we'll leave the planet."

And the engineers who actually work in space will tell you the joke: vacuum is a terrible place to cool a data center. No air, no convection — you radiate heat or you cook. The New Stack ran the skeptical piece the same day, and they're right. The orbital thing is a moonshot. But the fact that it's on a roadmap at all tells you how desperate the terrestrial situation has become.

What This Means for Independent Hosting Providers

Alright. You're not Nvidia and you're not SpaceX. You're running a colo, a hosting shop, or a small cloud. Here's what today changes for you:

First — start pricing by token, not by GPU. The market is moving to token economics. If you can't tell a customer what inference costs per delivered token on your hardware, you're flying blind. Public benchmarks like Artificial Analysis are becoming the pricing signal of this industry. Watch them.

Second — expect training silicon to loosen up. As the center of gravity shifts to inference, the secondary market for training-heavy GPUs is going to get softer. If you've been waiting to buy capacity, the window is opening — but don't fill your racks with decode-slow iron and expect to win inference workloads.

Third — optimize for memory bandwidth and latency, not FLOPS. Agentic workloads are coming to your customers whether you're ready or not. Long context, tool calls, chains of reasoning — these hammer the KV cache and the interconnect. If your architecture can't do low-latency decode, the hyperscalers and the Nebiuses of the world will eat your lunch on the workloads that actually pay.

Fourth — don't chase the orbital hype. Space data centers are a decade out and probably a fantasy. The real bottleneck — power, cooling, community consent — is still a terrestrial problem, and that's where you can win. Be the provider who solves the boring Earth problems while the giants go to space.

The Structural Reality

This pivot isn't a blip. Agentic AI generates more tokens per interaction than any chatbot era ever did, and every one of those tokens has to be decoded somewhere. The demand curve for inference is structural — it grows with adoption, not with hype cycles. And Nvidia has now positioned itself at every layer of that growth: the GPUs, the LPUs, the network, the software, and soon the orbit.

You don't pay $20 billion for a rival's architecture because you're feeling generous. You do it because the game changed, and you need the right tool for the next ten years — not the one that won the last ten.

The Bottom Line

Here's the truth bomb. Nvidia isn't selling chips anymore. It's selling the toll booth on every token ever generated, anywhere — on Earth, in a data center, and apparently in low Earth orbit. And the most honest thing it did all year was put a rival's name on the door.

That should scare you a little if you're competing on raw hardware. And it should motivate you a lot if you're competing on being smarter, faster, and closer to the customer than the giants can ever be.

Watch Wednesday's earnings call. But don't watch it for the revenue number. Watch it for what Jensen says about inference, tokens, and the factory. That's where the next decade is being priced.

— Allan Ali, Founder

This article was produced with AI-assisted research and editorial support. Sources: NVIDIA Newsroom (August 24, 2026); Network World; The New Stack; Spheron.