Ankur Duggal argues that checking an agent's final answer by feel is useful for getting started but insufficient for dependable production behavior. Traces reveal model calls, retrieved information and tool choices, allowing developers to connect failures with evaluations, human annotations and the application code.
Ankur Duggal demonstrates a flawed financial-information agent that gives inappropriate recommendations, fabricates answers and repeats or mishandles tool calls. A coding agent examines the traces and existing evaluations, proposes targeted checks, and builds datasets from failure cases rather than guessing from the final output alone.
Ankur Duggal uses a previous cross-market lookup experiment to show why one fix may be insufficient: normalizing a country name did not resolve an outdated tool dependency. He distinguishes creating evaluations from improving the underlying agent, stresses human review of changes, and explains that evaluation models, sampling and costs must fit the application's own use case.
Watch on YouTube
