Richard Diehl Martinez introduces Pico, a language-model research toolkit he developed during his PhD, in conversation with Pierce Freeman. It pairs pico-train with pico-analyze so researchers can study a model's development rather than examining only its final performance.
The training framework offers an editable decoder and hooks for retaining weights, gradients and activations at checkpoints. The analysis side applies metrics such as proportional effective rank, representation similarity and sparsity to investigate how different parts of a network change during training.
Rich discusses research on lower-resource Philippine languages and invites contributions to the analysis toolkit. He also sketches a longer-term ambition: using diagnostics during a training run to inform interventions. That adaptive feedback loop is a proposed direction, not a demonstrated automatic route to continuous learning or AGI.
Watch on YouTube
