Pico: Building Language Models You Can Analyze During Training

The Pretrained Pod16m 55s
Pierce Freeman in an off-white shirt gestures beside a smiling Richard Diehl Martinez in blue against black, beneath the white and blue headline 'SEE HOW YOUR MODEL LEARNS'.
0 comments · 0 votesOpen discussionClose discussion
Sign in to join the discussion

    Richard Diehl Martinez introduces Pico, a language-modelA language model learns patterns in sequences so it can interpret context and generate likely continuations such as text, code or structured responses. research toolkit he developed during his PhD, in conversation with Pierce Freeman. It pairs pico-train with pico-analyze so researchers can study a model's development rather than examining only its final performance.

    The training framework offers an editable decoder and hooks for retaining weights, gradients and activations at checkpoints. The analysis side applies metricsEvaluation is the systematic assessment of a system's behaviour or results against explicit criteria. such as proportional effective rank, representation similarity and sparsity to investigate how different parts of a network change during training.

    Rich discusses research on lower-resource Philippine languages and invites contributions to the analysis toolkit. He also sketches a longer-term ambition: using diagnostics during a training run to inform interventions. That adaptive feedback loop is a proposed direction, not a demonstrated automatic route to continuous learning or AGI.

    Original YouTube thumbnailWatch on YouTube

    Video topic coverage

    • Agents & workflows0
    • Coding & tools55
    • Models & learning90
    • Infrastructure & compute15
    • Research & evaluation100
    • Safety & security0
    • Business & productivity0
    • Creative & media0
    • Robotics & perception0
    • Society & policy15

    Share this video

    Pierce Freeman in an off-white shirt gestures beside a smiling Richard Diehl Martinez in blue against black, beneath the white and blue headline 'SEE HOW YOUR MODEL LEARNS'. With WWW.ARTIFICIAL-INTELLIGENCE.VIDEO, 21 February 2026 and duration 16m 55s.

    Pico pairs a minimal language-model training framework with analysis tools for weights, gradients and activations. Its creator explains why rich checkpoints can make learning dynamics easier to investigate.