What is instruction following?

Definition

AI instruction following measures whether a model acts on the task that was actually requested. It includes respecting format, scope, constraints, priorities and explicit requirements rather than merely producing related content.

Reliable adherence matters in real workflows because missed requirements create review and repair work. Evaluation should test competing instructions, long contexts and tool use while distinguishing valid user directions from unsafe or unauthorized requests.

ELI5

Instruction following is an AI model's ability to do the task that was actually requested while respecting its requirements. A related answer is not enough if it ignores the required format, scope, or constraints.

For example, if a user asks for three bullet points under 20 words each, a long essay fails the instruction even when its facts are correct. Good tests also check conflicting directions and unsafe requests.

Acronyms and aliases

model instruction adherence synonymAI instruction following variant

Frequently asked questions

How is AI instruction following evaluated?

Tests check whether outputs satisfy stated requirements, formatting, constraints and priorities across representative tasks.

Should a model follow every instruction?

No. It should follow valid authorized instructions while respecting higher-priority safety, privacy and permission boundaries.

Videos explaining instruction following

  1. Theo Browne beside the words Two Models Pull Ahead
    Theo Browne Ranks the Current AI Model Field
    Theo36m 35s1 VIEW
  2. A flat lightning bolt illustration beside the words Fast AI Gets Capable