Warp founder Zach Lloyd contrasts interactive prompting on individual laptops with a software factory in which people and agents participate in a managed development workflowWorkflow automation uses software or AI to complete a connected sequence of routine steps with less manual effort.. He argues that sandboxingAn AI agent sandbox is an isolated execution environment that limits which files, processes, networks, credentials, and external systems an agent can access., governance and an infrastructure definition in code can help teams operate those workflows.
Zach Lloyd demonstrates how teams can capture agent activity, define workflows and use evaluation scores to find recurring problemsEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria.. He describes a feedback loop in which agents analyze failures and propose changes to the factory configuration.
Zach Lloyd recommends benchmarking models against representative tasksA benchmark is a defined test or collection of tasks used to compare a system's performance under specified conditions. from a team's own work and comparing quality with cost. In the question session, he discusses specification-based testing, user-facing verification and the limitations of using an AI model as a judge.
Watch on YouTube
