Artificial intelligence model parallelism is used when a model or required throughput cannot be served efficiently on one accelerator. Tensor, pipeline and other strategies divide different parts of the model computation across devices.
Parallelism increases available memory and compute but introduces communication overhead. Performance therefore depends on partitioning, batch shape and the speed and topology of the network connecting accelerators.
Acronyms and aliases
AI model parallelism acronymartificial intelligence model parallelism variantmodel-parallel inference variant
Related terms
Frequently asked questions
How is model parallelism different from data parallelism?
Model parallelism divides one model across devices, while data parallelism places model replicas on devices that process different requests or examples.
Why does model parallelism need fast networking?
Devices must exchange intermediate activations or partial results during computation, and slow transfers can erase the benefit of additional processors.
