How Agent Observability Differs from Traditional Observability

AI Engineer20m 43s
Phil Hetzel beside the headline Agent Observability.
0 comments · 0 votesOpen discussionClose discussion
Sign in to join the discussion

    Phil Hetzel of Braintrust contrasts traditional service observability, such as latency and errors, with the additional challenge of assessing the quality of non-deterministic agent behaviourAI agent observability makes an agent's state, actions, tool use, failures, resource use, and outcomes visible enough to understand and operate it..

    He discusses inspecting rich traces, using human judgement and evaluations, and finding patterns in large collections of agent interactions. He describes how different teams need different views of the same system.

    Hetzel presents clustering and annotation as ways to understand failures and improve evaluation, while emphasising the practical challenges of monitoring complex agent systems.

    Original YouTube thumbnailWatch on YouTube

    Video topic coverage

    • Agents & workflows100
    • Coding & tools54.6
    • Models & learning0
    • Infrastructure & compute81.4
    • Research & evaluation90.7
    • Safety & security29.9
    • Business & productivity0
    • Creative & media0
    • Robotics & perception0
    • Society & policy0

    Share this video

    Phil Hetzel beside the headline Agent Observability. With WWW.ARTIFICIAL-INTELLIGENCE.VIDEO, 28 May 2026 and duration 20m 43s.

    Phil Hetzel explains how teams can observe and evaluate AI agents beyond traditional operational metrics.