What is a graphics processing unit communication bottleneck?
Definition
A graphics processing unit communication bottleneck appears when devices spend substantial time transferring data or waiting for other devices. The limiting link may be local device memory, an in-machine interconnect, a peripheral bus, or the network between cluster nodes.
Mitigation includes reducing communicated data, changing the partition, overlapping transfer with computation, using faster collectives, and matching operations to the correct link. Optimization must measure the actual hierarchy because different transfer paths have very different latency and bandwidth.
Acronyms and aliases
accelerator communication bottleneck variantGPU communication bottleneck variant
Related terms
Frequently asked questions
How can teams identify a graphics processing unit communication bottleneck?
Profiling can show devices waiting on transfers or synchronization and compare time spent communicating with time spent performing useful computation.
How can graphics processing unit communication costs be reduced?
Teams can communicate less data, improve partitioning, overlap transfer and computation, select efficient collectives, and use the fastest appropriate link.
Videos explaining graphics processing unit communication bottleneck
