Data and Environment Curation for Post-Training LLMs - Mahesh Sathiamoorthy, Bespoke Labs

AI Engineer19m 12s
Mahesh Sathiamoorthy in a blue top gestures beside the blue and white headline Post-Training Starts With Data on black.
0 comments · 0 votesOpen discussionClose discussion
Sign in to join the discussion

    Mahesh Sathiamoorthy describes the shift from testing what language models know to testing what agents can do over longer periods. He argues that reliability is a central obstacle and that high-quality examples and reinforcement-learning environmentsReinforcement learning trains an AI system to choose actions using feedback about the results of its earlier choices. are a bottleneck for post-trainingPost-training adapts a pretrained AI model to improve its behavior, capabilities, or alignment after the main training stage., even when models and compute are available.

    For reasoning data, Mahesh Sathiamoorthy outlines a pipeline of choosing source questions, mixing and filteringData curation selects, organizes, documents and maintains data so it is suitable for a defined use. them, generating answers with teacher modelsKnowledge distillation trains a smaller or different AI model to reproduce useful behavior learned from a stronger teacher model., and testing each choice through ablations. He reports that sampling several answers to one question can help and that a stronger teacher model is not always the better choice. He then extends the same empirical approach to agent trajectories and environments.

    Mahesh Sathiamoorthy says supervised fine-tuningFine-tuning adapts a pretrained model by updating its parameters on examples chosen for a particular task or subject. contributed much of the improvement in an agent-training example, while reinforcement learning added further gains at higher compute cost. In a separate deployment example, he reports that tagging training examples helped a recommendation model follow content rules without copying misleading numeric patterns. He closes by connecting environment versioning, sandboxes, rollout checkpoints, and post-training methods into one workflow.

    Original YouTube thumbnailWatch on YouTube

    Video topic coverage

    • Agents & workflows90
    • Coding & tools0
    • Models & learning100
    • Infrastructure & compute40
    • Research & evaluation100
    • Safety & security25
    • Business & productivity0
    • Creative & media0
    • Robotics & perception0
    • Society & policy0

    Share this video

    Mahesh Sathiamoorthy in a blue top gestures beside the blue and white headline Post-Training Starts With Data on black. With WWW.ARTIFICIAL-INTELLIGENCE.VIDEO, 31 July 2026 and duration 19m 12s.

    Mahesh Sathiamoorthy explains how selecting reasoning examples and agent environments can improve post-training reliability, with supervised fine-tuning sometimes delivering gains before costly reinforcement learning.