What is corrigibility?

Definition

A corrigible system lets its operators revise its objective or interrupt its work when new information appears. For example, a warehouse robot could accept a supervisor's stop command even if stopping delays its assigned delivery.

Corrigibility concerns how the system behaves when its instructions or operation change. Designers need to evaluate how it responds to intervention under different conditions, including when an interruption conflicts with completing a task.

ELI5

Imagine asking a helper to pack boxes and then noticing that the labels are wrong. You tell the helper to pause while you fix the labels.

A corrigible helper lets you interrupt and correct the plan. It does not keep packing just because finishing was the original goal.

Does corrigibility include accepting shutdown?

Authorized shutdown is one form of intervention that a corrigible system is designed to accept.

Why is correction important when a goal seems clear?

An operator may discover that the task is incomplete or that circumstances have changed. The system needs to allow an authorized update.

  1. Roman Yampolskiy and Stuart Russell discuss AI control beside the headline Can AI Stay Correctable on a black background.
  2. Blue and white “AGENT AUTONOMY THE LIMITS” headline beside a blue arrow stopping at a white boundary on black.
  3. Wes Roth beside the headline AGENT SAFETY in blue and white lettering on a black background.
    OpenAI's employee's WARNING
    Wes Roth24m 21s
Definition card for corrigibility: What is corrigibility?

Corrigibility is the property of a system that remains open to authorized correction, changes, or shutdown.