Nishant Gupta argues that long-running, stateful AI agents need infrastructure designed for unpredictable decisions and failures. He describes how invalid tool calls can produce runaway retries, rising compute use and outages.
He recommends separating model proposals from infrastructure validation and execution, with a control planeAn AI control plane observes AI activity and applies centralized policy without owning the application's primary task logic. for scheduling, policy, memory coordination and monitoring. Traces of tool calls and state transitionsAI agent observability makes an agent's state, actions, tool use, failures, resource use, and outcomes visible enough to understand and operate it. can make agent failures easier to investigate.
Gupta discusses shared-memory consistency, layered safeguards, human handling of exceptionsHuman-in-the-loop AI keeps a person involved in reviewing, correcting, approving, or guiding an AI system's work. and GPU scheduling. He frames circuit breakers, rate limits and resource quotasResource utilization measures how much of an available computing resource is being used over a defined period. It can describe processor activity, memory occupancy or other capacity relevant to an AI workload. as existing distributed-systems patterns that can be adapted to agent workloads.
Watch on YouTube
