Tejas Kumar distinguishes evaluationEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria. before release from protections applied by an agent harnessAn AI agent harness is the software framework that packages a model with tools, instructions, context management, execution controls, and user interaction. at runtime. He introduces test cases, expected behavior, judges and aggregation, then discusses position, verbosity, self-preference and sycophancy as reasons an automated judgeA large language model as a judge is an evaluation method in which a language model scores, compares or critiques another system's output using stated criteria. can give misleading results.
The live coding example starts with a refund-policy scenario, moves beyond a brittle text check and compares judge decisions with a labeled datasetHuman judgment in AI is the accountable interpretation and decision-making people contribute when setting goals, evaluating evidence, and managing consequences.. Kumar adds policy context, changes model when necessary and builds a CI agreement gate. He closes with retrieval-backed policy and fresh production examples; his numerical thresholds are choices for the demonstration rather than universal reliability guarantees.
Watch on YouTube




