What is inference latency?

Definition

AI inference latency measures how long a model takes to respond during use. Teams may track time to first token, time per generated token or total completion time depending on the application.

Latency depends on model size, quantization, active parameters, context length, hardware, memory bandwidth and software. A smaller quantization can be faster, but implementation and hardware support determine the actual result.

ELI5

AI inference latency is how long a model takes to respond after receiving input. Teams can measure time to the first token, time between later tokens or the full completion time.

For example, a chat assistant may start speaking quickly but pause between words because later token generation is slow. Model size, context, hardware, memory bandwidth, software and current load all affect the result.

Acronyms and aliases

model response latency synonymAI inference latency variant

Frequently asked questions

What affects AI inference latency?

Model architecture, precision, active parameters, context, output length, hardware, memory bandwidth and software all affect it.

Does lower-bit quantization always reduce latency?

No. It often helps, but unsupported kernels, conversion overhead or hardware characteristics can limit the speedup.

Videos explaining inference latency

  1. Ornith 1.5 35B Q4 vs Q8
    Bijan Bowen31:401 VIEW