Mahesh Sathiamoorthy describes the shift from testing what language models know to testing what agents can do over longer periods. He argues that reliability is a central obstacle and that high-quality examples and reinforcement-learning environmentsReinforcement learning trains an AI system to choose actions using feedback about the results of its earlier choices. are a bottleneck for post-trainingPost-training adapts a pretrained AI model to improve its behavior, capabilities, or alignment after the main training stage., even when models and compute are available.
For reasoning data, Mahesh Sathiamoorthy outlines a pipeline of choosing source questions, mixing and filteringData curation selects, organizes, documents and maintains data so it is suitable for a defined use. them, generating answers with teacher modelsKnowledge distillation trains a smaller or different AI model to reproduce useful behavior learned from a stronger teacher model., and testing each choice through ablations. He reports that sampling several answers to one question can help and that a stronger teacher model is not always the better choice. He then extends the same empirical approach to agent trajectories and environments.
Mahesh Sathiamoorthy says supervised fine-tuningFine-tuning adapts a pretrained model by updating its parameters on examples chosen for a particular task or subject. contributed much of the improvement in an agent-training example, while reinforcement learning added further gains at higher compute cost. In a separate deployment example, he reports that tagging training examples helped a recommendation model follow content rules without copying misleading numeric patterns. He closes by connecting environment versioning, sandboxes, rollout checkpoints, and post-training methods into one workflow.
Watch on YouTube




