Samuel Denton organizes continual learning around two axes: whether production traces are offline or online, and whether the hints used to improve a student model are static or generated from the model's current rollout. Combining the axes creates four practical distillation patterns for enterprise AI agents.
Samuel Denton focuses on two ends of that framework. Offline hints applied to stored production traces can target useful behavior immediately, while online hints generated from live rollouts can continuously raise a model's performance ceiling without assuming a perfect answer exists for every task.
Samuel Denton reports that offline distillation increased a coding agent's task-completion behavior without reducing its test pass rate, while online hints sharply improved an unusual hyperlink-formatting requirement. He recommends placing hints near the relevant decision and masking teacher tokens that do not support the target behavior.
Watch on YouTube
