What is mechanistic interpretability?

Definition

Mechanistic interpretability examines model internals such as parameters, activations, learned features, and computational pathways. Researchers use experiments and analytical tools to connect those internal patterns with concepts, decisions, and observed outputs.

The goal is deeper than describing a model's input and output correlation. Useful evidence should support causal tests, distinguish genuine mechanisms from convenient stories, and show whether an internal intervention changes behavior in the predicted way.

ELI5

Mechanistic interpretability tries to understand what is happening inside an AI model while it works. It looks for internal patterns that help explain why the model produced a particular answer or action.

For example, researchers can find an internal pattern that responds to a concept, adjust it, and check whether the model's behavior changes as expected. That experiment provides stronger evidence than a guess based only on the final answer.

Frequently asked questions

What does mechanistic interpretability inspect?

It can inspect parameters, activations, features, circuits, attention patterns, representations, and the causal pathways that connect model inputs with outputs.

Why are causal interventions important?

Changing an internal component and observing the predicted behavioral effect helps show that the component contributes to the behavior rather than merely correlating with it.

Videos explaining mechanistic interpretability

  1. Tim Scarfe and Tom McGrath beside the words Design AI from the Inside