Ben Holmes explains how teams can collect agent conversations, tool calls and outputsAn execution trace records the sequence of operations and results produced while a system carries out a task. to examine work that might otherwise appear successful despite hidden retries or failures.
Ben Holmes describes using language models with explicit rubrics to score qualitiesEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria. such as efficiency and test coverage. Aggregated failures can guide an agent to propose changes to skillsAn AI agent skill is a reusable package of instructions, resources, and tool guidance for performing a bounded kind of work. or configuration, which people review.
Ben Holmes demonstrates a one-time analysis and describes tracking scores over time. His examples are a practitioner account of the workflow rather than an independent evaluation.
Watch on YouTube
