What is artificial intelligence compute infrastructure?
Definition
Artificial intelligence compute infrastructure includes processors, memory, storage, networking, model runtimes, schedulers, and monitoring systems. For inference, it must place models on suitable hardware, route requests, manage concurrency, and deliver predictable latency and throughput.
Infrastructure may be centralized in a cloud data center, deployed on private equipment, or distributed across independent providers. Each model changes the balance of cost, control, reliability, privacy, and operational complexity. Capacity claims should be evaluated against measured workloads rather than hardware totals alone.
Acronyms and aliases
AI compute infrastructure variantAI infrastructure variant
Specialised terms
Related terms
Frequently asked questions
What does artificial intelligence compute infrastructure include?
It includes processors, memory, storage, networks, model-serving software, schedulers, security controls, monitoring, and the operations needed to keep them available.
How is inference infrastructure different from training infrastructure?
Training emphasizes sustained large-scale computation and data movement, while inference emphasizes request routing, latency, throughput, model availability, and cost per result.
