Samuel Denton organizes continual learningContinual learning lets an AI system acquire new knowledge or skills over time while retaining what it learned earlier. around two axes: whether production tracesAn execution trace records the sequence of operations and results produced while a system carries out a task. are offline or online, and whether the hints used to improve a student model are static or generated from the model's current rollout. Combining the axes creates four practical distillationKnowledge distillation trains a smaller or different AI model to reproduce useful behavior learned from a stronger teacher model. patterns for enterprise AI agents.
Samuel Denton focuses on two ends of that framework. Offline hints applied to stored production traces can target useful behavior immediately, while online hints generated from live rollouts can continuously raise a model's performance ceiling without assuming a perfect answer exists for every task.
Samuel Denton reports that offline distillation increased a coding agent's task-completion behavior without reducing its test pass rate, while online hints sharply improved an unusual hyperlink-formatting requirement. He recommends placing hints near the relevant decision and masking teacher tokens that do not support the target behavior.
Watch on YouTube
