Time to first token includes queueing, routing and the prompt prefill stage before output generation begins. It is a key measure of perceived responsiveness for interactive applications because the user sees no model output during this interval.
Caching a repeated prefix can reduce time to first token by avoiding prompt recomputation. Capacity planning, prefill scaling, model placement and request scheduling also influence the delay.
Acronyms and aliases
TTFT acronymresponse startup latency synonym
Related terms
Frequently asked questions
What does TTFT stand for?
TTFT is the acronym for time to first token, the startup delay before the first generated token arrives.
Does a low time to first token guarantee fast generation?
No. Startup can be fast while later tokens arrive slowly, so inter-token latency and total response time must also be measured.
