What is sample efficiency?

Definition

A sample-efficient learner reaches a target level of performance with fewer observations, demonstrations or interactions. It can use prior knowledge, strong abstractions or better learning rules so each new example changes behavior more effectively.

Efficiency matters when data is expensive, rare or risky to collect. It should be measured against the same task and quality target because a model that uses fewer samples but performs worse has not necessarily improved the practical tradeoff.

ELI5

Sample efficiency describes how much an AI can learn from a small amount of experience. A more sample-efficient system needs fewer examples to reach the same useful skill level.

For example, a person may understand a new object category after seeing a few examples while a model needs thousands. Improving sample efficiency could reduce data collection and training cost, but the learned skill still needs the same quality tests.

Frequently asked questions

Why is sample efficiency important for AI?

It can reduce data, time and cost, especially when examples require experts, physical trials or rare events.

How should sample efficiency be compared?

Compare systems on the same task, data accounting and target performance rather than counting examples alone.

Videos explaining sample efficiency