Part 1: Accelerate your training speed with the FSDP Transformer wrapper
Want to learn how to accelerate your transformer model training speed by up to 2x+? The transformer auto-wrapper helps FSDP better understand your model’s optimal wrapping points. This can accelerate your training throughput by 2x+ vs the default wrapper. Learn how to implement it in under 5 minutes!