Nick Nisi on Measuring Agent Skills and Workflows

AI Engineer17m 43s
Nick Nisi beside the blue and white headline Measure Agent Skills on black.
0 comments · 0 votesOpen discussionClose discussion
Sign in to join the discussion

    WorkOS developer experience engineer Nick Nisi describes an internal agent harness with implementation, verification, review and retrospective stepsAn AI agent harness is the software framework that packages a model with tools, instructions, context management, execution controls, and user interaction.. He says gates and evidence help him review work before accepting it.

    Nisi recounts replacing a large generated set of product instructions with a much smaller set of common pitfalls. He reports that his own evaluations became faster and more successful, including one case where adding a skill reduced the pass rateEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria..

    He recommends measuring agent behavior, making requirements enforceable and retaining useful lessons from failures. His examples reflect his own workflow and are not a universal comparison of agent systems.

    Original YouTube thumbnailWatch on YouTube

    Video topic coverage

    • Agents & workflows100
    • Coding & tools86.9
    • Models & learning0
    • Infrastructure & compute0
    • Research & evaluation87.9
    • Safety & security26.3
    • Business & productivity0
    • Creative & media0
    • Robotics & perception0
    • Society & policy0

    Share this video

    Nick Nisi beside the blue and white headline Measure Agent Skills on black. With WWW.ARTIFICIAL-INTELLIGENCE.VIDEO, 30 May 2026 and duration 17m 43s.

    Nick Nisi shares lessons from enforcing evidence gates and measuring a smaller set of agent instructions.