The Christmas Eve agreement—billed as Nvidia’s biggest deal in its three-decade history—landed at a precarious moment for Groq. Now Nvidia is betting on Groq’s inference-speed tech inside a newly announced chip platform.

By Phoebe Liu, Forbes Staff. With reporting from Iain Martin, Rich Nieva and Anna Tong.

Last winter, Groq cofounder and CEO Jonathan Ross walked into a meeting with Nvidia CEO Jensen Huang with a pitch for the companies’ tech to work together. He now describes the synergy with a logistics analogy: stop building AI data centers as if every workload wants the same hardware. Training is bulk hauling; inference is last-mile delivery. GPUs can do both, but using the 18-wheeler even when you just need a van can be a lot slower. So: Nvidia’s general-purpose GPUs are the big trucks. Groq’s specialized chips—LPUs, or language processing units, designed to run models fast—are the smaller vans. “If you were building out a logistics network for the entire United States, and I told you your two options were all 18-wheelers or just delivery vans, which one would you pick?” Ross said. “The best answer is both.”

Ross wasn’t just pitching a worldview. He wanted Nvidia’s permission to buy around 100,000 Blackwell chips, likely worth billions. Huang grilled him on the technical details, and then the call ended.

 When Huang called back three days later, Ross expected a discussion about his GPU purchase order. Instead, the Nvidia CEO cut to the chase. “We should probably move really fast,” Ross recalled him saying.

Three weeks later, Nvidia announced a $20 billion Christmas Eve deal to license Groq’s product and hire most of its staff. In Silicon Valley terms, it read like a merger without the paperwork: take the team, secure the tech, and get the strategic benefit without inheriting every loose end in the corporate attic, or running into antitrust issues. (Groq’s remaining independent company, an LPU cloud provider, still exists and is growing, Ross—now Nvidia’s chief software architect—told Forbes; former employees said last month the company is expected to sell but hasn’t yet.)

At the time, it didn’t make sense to everyone—because it wasn’t obvious what Nvidia wanted with Groq beyond a very expensive “we’re serious about inference” press cycle. Even Huang’s own explanation stoked bafflement. “They had a very hard time addressing the mainstream part of AI factories,” he said at a conference earlier this year. “But in combination with us, they don’t have to.”

But three months later, Nvidia made their........

© Forbes