Arena co-founder Anastasios Angelopoulos explains a user-based agent evaluationEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria. that compares models on signals such as task completion, praise, complaints and steerability. He describes results for a newly released model and cautions that its strengths differ across these measures.
Angelopoulos and the interviewer discuss refusal behavior, long-running tasks and how shifting user expectations affect comparisons. He describes the challenge of protecting systems against adversarial useAI safety is the field and practice of reducing harmful failures, misuse, loss of control, and unintended consequences from AI systems. without making them frustrating for ordinary users.
The interview also covers changing patterns of delegationAI agent delegation assigns a bounded task, context, authority, and expected result from one participant to an agent or another agent. and the limits of judging capability from one score. Model comparisons and causal explanations are presented as the speakers views and the reported evaluation results.
Watch on YouTube
