What is a vision-language-action model?

Definition

A vision-language-action model receives images or video together with a language goal and produces an action representation. It aims to connect what the robot sees, what the person wants and how the robot should respond within one learned model.

The model still operates inside a larger robotics stack. Device controllers, safety checks and execution feedback remain important because a plausible action prediction can be physically impossible, poorly timed or unsafe in the current environment.

ELI5

A vision-language-action model helps a robot connect seeing, understanding words and acting. It tries to answer three questions together: what is in front of me, what did the person ask, and which supported action should I take?

For example, a person might say, pick up the red block, while the robot sees several objects. The model can connect the words with the correct object and propose an action, but the robot still needs control and safety systems to carry it out reliably.

Frequently asked questions

What inputs does a vision-language-action model use?

It typically uses visual observations, language instructions and information about the robot or task state.

Does a vision-language-action model replace robot safety controls?

No. Its action predictions still need device-specific control, validation, permissions and safety enforcement.

Videos explaining vision-language-action model