AI compute capacity combines the quantity and capability of processors with memory, networking, power, cooling, software, and usable operating time. Capacity is meaningful only when the system can schedule workloads and keep the hardware productively utilized.
Organizations obtain capacity by owning infrastructure, renting cloud services, or signing longer-term contracts. Effective capacity can change as new chips improve performance, models become more efficient, power becomes constrained, or demand shifts between training and inference.
ELI5
Compute capacity is the amount of AI work a collection of hardware and supporting systems can perform over time. It depends on processors, memory, networking, power, cooling, software, and how effectively jobs are scheduled.
For example, a company may own many graphics processors but have limited usable capacity if power shortages keep half of them offline. Faster models or better scheduling can increase effective capacity without adding another device.
Is AI compute capacity the same as the number of graphics processors?
No. Processor count matters, but memory, networks, power, software, availability, and utilization determine the usable capacity.
Why do AI laboratories reserve compute capacity?
Reservations can guarantee access for planned training and inference workloads when suitable infrastructure is scarce or slow to build.
