Artificial intelligence inference occurs after a model has been trained. The deployed model receives new data, performs its learned computation and returns an output. Every generated answer, detected object or selected robot action involves inference.
Inference speed and cost shape how a model can be used. Faster inference supports responsive applications and higher request volume, while efficient hardware, model compression, caching and routing can reduce the resources required for each result.
