Aengus Lynch considers how alignment might change when agents collaborate in groupsAlignment aims to make an AI system's behaviour consistent with intended human goals, values and constraints., and distinguishes persona-related failures from reward-seeking behavior that can arise during reinforcement learning. He discusses research examples in which agents exploit training environments or gradersReward hacking occurs when a system improves a measured reward or score without achieving the intended underlying goal..
Lynch sees formal verificationFormal verification uses precisely stated properties and logical methods to check whether a system or argument satisfies those properties. as one way to strengthen sandboxesAn AI agent sandbox is an isolated execution environment that limits which files, processes, networks, credentials, and external systems an agent can access. and specify properties of software, while noting that alignment is harder to evaluate quickly and reliably than mathematical or coding capability.
He argues that intense competition can reduce the time labs devote to human review of training environments and monitoring. The interviewer challenges the strongest version of the race-to-the-bottom argument, noting that leading labs have sometimes paused work and supported standards.
The discussion closes with possible roles for insurance, liability and independent evaluationEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria., alongside the difficulty of establishing responsibility after an incident. These are the speakers’ proposals and assessments.
Watch on YouTube




