Agentic AI inference runs the model calls produced as an agent reasons, uses tools and returns to a task over many turns. These sessions can reuse a large system prompt while adding much more input context than output, so their resource pattern differs from a short independent chat request.
Efficient serving needs coordinated routing, cache management, scaling and hardware-aware scheduling. The system must preserve useful context when possible while still meeting interactive latency goals under traffic that changes across sessions.
ELI5
Agentic inference is the repeated model work needed while an AI agent carries out a longer task. The agent may ask the model what to do, use a tool, examine the result, and ask the model again many times.
For example, a research agent can search for a document, read it, compare it with another source, and draft a report through several model calls. The serving system must keep the useful context available without making every step slow or expensive.
