What is model scaling?

Definition

Artificial intelligence model scaling is the practice of increasing one or more resources used to build or run a model. Common dimensions include parameter count, training data, training computation and the amount of computation used during inference.

Scaling can improve performance, but it is not the only source of progress. Adaptation methods, tools, data quality and experimental design may matter just as much on a particular task, so claims about scale should identify what changed and how it was measured.

Acronyms and aliases

model scaling synonymAI model scaling variantartificial intelligence model scaling variant

Frequently asked questions

Does a larger AI model always perform better?

No. Larger models often improve on some measurements, but results also depend on data quality, training objectives, inference methods and the task. Size alone does not guarantee reliable or general capability.

What resources can be scaled in an AI model?

Developers may scale parameters, training examples, training computation, context length or inference-time computation. Each dimension has different costs and can affect performance differently.

Videos explaining model scaling