Dylan Bristot describes Nebius’s managed inference service and its proposed loop from production data and post-trainingPost-training adapts a pretrained AI model to improve its behavior, capabilities, or alignment after the main training stage. to deployment. His presentation includes company product claims.
Sujee Maniyam discusses selecting serving engines and routing requests toward cached contextAI model routing sends each request to a model chosen for that request's complexity, cost, speed, privacy, or other requirements.. He explains speculative decodingSpeculative decoding accelerates text generation by proposing several tokens cheaply and checking them together with the target model., where a smaller model proposes tokens that a larger model verifies.
He also discusses cache offloading, separating prefill and decoding, and quantizationModel quantization represents AI model values with fewer bits to reduce storage, memory use and often inference cost. tradeoffs. Speedup figures are attributed to Nebius and depend on the workload and deployment.
Watch on YouTube
