What is preference optimization?

Definition

Preference optimization trains a model from signals that compare outputs instead of requiring one exact target answer for every input. Feedback may rank two candidates, assign scores or indicate which behavior better satisfies quality and safety criteria.

The method is often used after a model already has the underlying capability. It can reduce recurring failure patterns, such as repetition, omitted content or degenerate audio, by making preferred outputs more likely relative to poor ones.

Preference data must represent the intended users and task. Biased, inconsistent or overly narrow judgments can shift the model toward superficial preferences or weaken useful behavior that the feedback did not measure.

ELI5

Preference optimization teaches a model by showing which of two answers is better. The model learns to make future outputs more like the preferred examples.

For example, reviewers can prefer a complete spoken sentence over one that repeats a phrase or skips words. The training then discourages those specific mistakes without teaching speech from the beginning.

Frequently asked questions

Does preference optimization teach a model a new capability?

It is usually most effective for shaping and improving a capability the model already has rather than creating that capability from nothing.

Where does preference data come from?

It can come from human comparisons, carefully designed automated evaluators or a mixture of both, subject to quality checks.

Videos explaining preference optimization

  1. Pavan Muddireddy and Tim Scarfe beside an audio waveform flowing into a responsive AI assistant interface on a black background.