Ronak Malde argues that continual learningContinual learning lets an AI system acquire new knowledge or skills over time while retaining what it learned earlier. should use the enormous stream of production inference tracesAn execution trace records the sequence of operations and results produced while a system carries out a task. rather than depend on increasingly expensive benchmark environments. Current approaches suffer from mismatched task distributions, off-policy sampling, costly parallel rollouts and sparse sequence-level rewards.
The proposed on-policy self-distillationKnowledge distillation trains a smaller or different AI model to reproduce useful behavior learned from a stronger teacher model. method turns the student model into its own stronger teacher by adding privileged guidance to the teacher prompt. Training then matches the unguided student's token probabilities to the guided teacher, preserving on-policy data while extracting dense feedback from a single rollout instead of a large rollout group.
Short tasks benefit from better token efficiency and broader shifts in the output distribution, but long tool-using trajectories expose serious failures. When student and teacher paths diverge, the teacher repeatedly favors hedging words and pulls the model into a poor local compromise; step-level divergence weighting limits training to portions of the trajectory where correction remains meaningful.
Privileged hints create a second risk by leaking answers that would not be available during real deployment. Filtering can convert direct solutions into realistic process guidance, while residual guidance compares partial and complete hints to avoid shifting the student toward behavior far outside its original distribution.
Together, these controls are presented as a scalable step toward systems that learn from online tasks with one rollout and per-token feedback. The closing vision is still provisional: present systems rely partly on offline batch updates, and genuinely continuous co-adaptation between models and their agent harnessesAn AI agent harness is the software framework that packages a model with tools, instructions, context management, execution controls, and user interaction. remains an open research and infrastructure problem.
Watch on YouTube
