What is a learned feature?

Definition

A learned feature can respond to a recurring property in data, such as a concept, style, grammatical role, visual shape, behavior, or abstract relationship. Features may be distributed across many internal units or appear as directions in an activation space.

A discovered feature needs careful validation because the examples that activate it may support several interpretations. Researchers test additional inputs, measure selectivity, trace downstream effects, and intervene on the feature to determine what it actually contributes.

ELI5

A learned feature is a pattern an AI model develops inside itself because the pattern helps with its work. It can represent something concrete or abstract, but it may not match a neat human label.

For example, an internal feature might respond strongly to text about promises. Researchers test many different sentences to learn whether it represents promises specifically or a broader idea such as obligation.

Frequently asked questions

Where can a learned feature appear?

It can appear in one internal unit, across many units, as a direction in activation space, or as part of a larger computational circuit.

Why can learned features be difficult to name?

A feature may respond to several correlated properties, combine multiple concepts, or represent a distinction that does not have a simple human label.

Videos explaining learned feature