When an AI company goes shopping for a GPU cloud, there are two questions it asks. “Can you get me the chips right now?” and “Will they still be working tomorrow?”

Price, surprisingly, is the third consideration. Orchestration software—the layer deciding which job goes to which GPU—falls even further down the list. 

“If tomorrow our product team says, ‘We just signed a telecom customer, and we need another 256 H100sH100Nvidia’s H100 is a data centre-grade GPU by Monday’,” said an ML engineer at the voice-AI startup, Gnani,........

© The Ken