Jacob Effron interviews Nebius CTO Danila Shtan about building an AI cloud, the company’s origins and the engineering work needed to integrate new GPU generations. Shtan describes a software-led approach to serving customers at different scales.
They discuss storage throughput, inference latencyInference is the use of a trained model to produce an output from supplied input., disaggregated workloads and model optimization. Shtan describes differences between research labs and enterprise customers and argues that execution across supply and infrastructure constraintsAI compute infrastructure is the hardware, facilities, networks, power, cooling, storage, and software used to train and run AI models. is a major bottleneck.
Shtan attributes independent cloud providers’ opportunity partly to chip scarcity and competing internal demand at larger companies. He expects hyperscaler competition to grow, and discusses responsible engineering practices and reliable agent outcomes.
Watch on YouTube
