Dwarkesh Patel argues that session notes and external memoryAI agent memory is stored information that an agent can retrieve and use across steps, sessions, or changing contexts. are not enough for durable skill accumulation. If deployed models begin updating their own weights from work experience, safety evaluationEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria. would need to shift from a single predeployment gate to recurring risk inspections that track a changing system.
Continual learningContinual learning lets an AI system acquire new knowledge or skills over time while retaining what it learned earlier. could create strong feedback loops for early deployers. Models with more users would gather more experience, improve faster and attract even more usersA network effect occurs when a product becomes more valuable to users as more people use it., increasing pressure on laboratories to release systems earlier and making established products harder to displace.
The same dynamic could deepen laboratory and enterprise lock-inVendor lock-in occurs when switching away from a provider becomes difficult or costly because systems depend on its proprietary interfaces, data, features, contracts, or operations.. Organizations may need to expose internal work data so models can learn effectively, while large users gain additional advantages from batching inference across many tasks. Patel's predictions frame continual learning as an economic and governance change, not only a technical improvement.
Watch on YouTube




