Multi-graphics processing unit computing uses several graphics processors to handle a workload that is too large or slow for one device. Software partitions data or model work, launches kernels on each processor, and exchanges intermediate results through local interconnects or cluster networks.
Scaling is not automatic because communication and synchronization can dominate runtime. Efficient designs choose a suitable partition, overlap transfer with computation, minimize unnecessary copies, and account for the different costs of links inside one machine and across machines.
