NVIDIA’s Ziv Ilan discusses latency in diffusion-based image and video generationLow-latency AI inference produces a model result quickly enough for an interactive or time-sensitive application. and describes quantization, caching and step distillation as complementary optimization approaches.
Ilan explains that quantization reduces numerical precisionModel quantization represents AI model values with fewer bits to reduce storage, memory use and often inference cost., while caching can reuse computation between denoising steps. He cautions that caching thresholds and quantization can affect output quality.
Step distillation trains a student model to produce outputs in fewer denoising stepsKnowledge distillation trains a smaller or different AI model to reproduce useful behavior learned from a stronger teacher model.. Ilan compares trajectory and distribution-based approaches and discusses the compute, data and evaluation trade-offs involved. His performance examples are attributed to the presentation.
Watch on YouTube
