Arena co-founder Anastasios Angelopoulos discusses model rankings with the MTS hosts, explaining that aggregate preferences can differ from expert and agent-task results. The discussion reflects Arena’s own evaluation methodsEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria. and commercial perspective.
Angelopoulos argues that the cost of completing a taskCost per completed task measures the total AI, tool, infrastructure, retry, and repair expense for each verified useful outcome. combines price with model behavior and success rate, so price per token alone can be misleading. The conversation also covers evaluation of visual output, writing style, security and personalization.
The interview considers shared contextContext engineering designs how relevant instructions, evidence, state and tool information are assembled for a model's task. and memory compactionAI agent memory is stored information that an agent can retrieve and use across steps, sessions, or changing contexts. across models, along with the difficulty of evaluating longer-term learning.
Watch on YouTube
