A graphics processing unit contains many execution units designed to process large groups of data in parallel. Although developed for graphics, the same architecture works well for matrix multiplication and other numerical operations used in neural-network training and inference.
Performance depends on more than arithmetic capacity. Memory bandwidth, kernel design, data movement, synchronization, and communication with other processors determine how much of the hardware's theoretical throughput a workload actually uses.
ELI5
A graphics processing unit is a processor with many small execution units that perform similar calculations in parallel. The design works well for the matrix operations used in AI training and inference.
For example, a GPU can calculate many parts of a neural-network layer at the same time instead of processing them one by one. Real speed also depends on memory bandwidth, software kernels and how efficiently data reaches the processor.
