What is model distillation?

Definition

Model distillation transfers behavior from a teacher model to a student model. The student can learn from teacher-generated labels, probability distributions, reasoning traces where permitted, synthetic examples, or other structured outputs instead of relying only on the original training data.

Distillation can reduce serving cost or create a model specialized for a narrower task, but it raises technical, contractual, and provenance questions. Access terms may restrict using one provider's outputs to train a competing model, and a student can inherit teacher errors or lose capabilities during compression.

ELI5

Model distillation is a way to teach one AI model using examples or guidance produced by another model. The goal is often to make a smaller, cheaper student copy useful parts of a stronger teacher's behavior.

For example, a teacher might label many difficult questions and a student model trains on those labeled examples. The student may become good at that task, but it can also copy mistakes, miss subtle skills, and violate access terms if the teacher's outputs were not permitted for training.

Acronyms and aliases

teacher-student distillation variant

Frequently asked questions

Is model distillation the same as copying model weights?

No. Distillation usually trains a separate student from teacher behavior or outputs rather than copying the teacher's internal parameters directly.

Why do model providers restrict distillation?

A provider may view large-scale training on its outputs as competitive replication, misuse of the service, or a breach of contractual limits and provenance requirements.

Videos explaining model distillation