Tom McGrath describes Goodfire's effort to turn mechanistic interpretabilityMechanistic interpretability studies the internal computations of AI models to explain how their components produce behavior. into practical infrastructure for understanding model internals. Instead of treating a neural network as an opaque collection of weights, the work looks for meaningful featuresA learned feature is an internal pattern an AI model develops to represent information that helps it perform its task., directions and geometric relationships inside its activationsA model activation is an intermediate numerical response produced inside an AI model while it processes an input..
McGrath explains that these representations can support controlled generalization and data debugging. Teams can identify which concepts drive a behavior, trace unexpected outputs back to training examplesTraining data attribution estimates which training examples most influenced a particular AI model output or behavior. and test interventions that change a model's internal stateActivation steering changes selected internal AI model activations to influence behavior during inference. without relying only on prompts.
The same tools may help evaluate hallucinations, reward hacking and other safety failures. McGrath stresses that useful interpretability must connect internal measurements with real behavior, while remaining careful about what a discovered feature actually represents.
Watch on YouTube
