Tips and tricks for distributed large model training

Discover several different distribution strategies and related concepts for data and model parallel training. Walk through an example of training a 39 billion parameter language model on TPUs, and conclude with the challenges and best practices of orchestrating large scale language model training.

TensorFlow website →

Speakers: Nikita Namjoshi, Vaibhav Singh

Watch more:
All Google I/O 2022 Sessions →
ML/AI at I/O 2022 playlist →
All Google I/O 2022 technical sessions →

Subscribe to TensorFlow →


Source of this TensorFlow AI Video

AI video(s) you might be interested in …