Jason Lopatecki describes observability as more than a dashboard for humans: tracesAn execution trace records the sequence of operations and results produced while a system carries out a task. and evaluation resultsEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria. can provide the context an agent needs to investigate failures. In the demonstrated workflow, scheduled or event-driven investigations prepare issues and possible fixes before an engineer begins the review.
The Arize examples combine repository code, retrieved telemetry, configurable skillsAn AI agent skill is a reusable package of instructions, resources, and tool guidance for performing a bounded kind of work. and a chosen agent harnessAn AI agent harness is the software framework that packages a model with tools, instructions, context management, execution controls, and user interaction. or sandboxAn AI agent sandbox is an isolated execution environment that limits which files, processes, networks, credentials, and external systems an agent can access.. Lopatecki distinguishes small fixes from changes that still need an engineer to drive them forward, and explains that online evaluations add useful signals to traces without replacing the underlying evidence.
Watch on YouTube




