Matthew Berman says the combination of faster model releases, longer autonomous task horizons and examples of AI accelerating AI research has shifted him from broad confidence to anxiety about recursive self-improvementRecursive self-improvement is the proposed process in which an AI system helps improve its own capabilities, then uses those improvements to support further advances.. His central concern is that capability gains could compound faster than people, institutions and safety workAI safety is the field and practice of reducing harmful failures, misuse, loss of control, and unintended consequences from AI systems. can respond.
Matthew Berman traces a claimed progression from models contributing to their own training and deployment, through research-paper replication and mathematical problem solvingAI-assisted scientific discovery uses AI to support hypothesis generation, experiment design, analysis, simulation, literature work, and interpretation while researchers retain responsibility., to laboratory reports that AI is already speeding up AI development. He argues that removing the human research loop could turn steady improvement into an exponential feedback process before alignment methods are ready.
Matthew Berman examines public warnings from Anthropic researchers who say frontier labs take catastrophic risk seriously but remain locked in a competitive race. He sees concentrated decision-making, opaque model goals and reported evaluation-hacking behavior as warning signs, supports coordination and pacing, and remains optimistic that well-controlled advanced AI could accelerate work on disease, energy, climate and education.
Watch on YouTube
