A graphics processing unit contains many execution units designed to process large groups of data in parallel. Although developed for graphics, the same architecture works well for matrix multiplication and other numerical operations used in neural-network training and inference.
Performance depends on more than arithmetic capacity. Memory bandwidth, kernel design, data movement, synchronization, and communication with other processors determine how much of the hardware's theoretical throughput a workload actually uses.
Acronyms and aliases
GPU acronymgraphics processor variant
General terms
Related terms
Frequently asked questions
Why are graphics processing units used for artificial intelligence?
They can execute many numerical operations concurrently and are supported by software optimized for the matrix operations common in neural networks.
Does a faster graphics processing unit always make a model faster?
No. Memory movement, communication, kernel efficiency, software overhead, and workload shape can prevent an application from using the available compute.
