General Intuition’s Pim de Witte argues that game recordings and controller inputsImitation learning trains an agent to perform a task by learning from demonstrations of behavior rather than relying only on an explicitly written reward function. can supply large quantities of action-labeled data for models that observe pixels and predict actionsComputer vision enables software to analyze visual inputs such as images or video and extract information about their contents or structure.. He contrasts this approach with turn-based language models and describes its potential for computer useComputer use lets an AI agent interact with software through interface controls such as screens, buttons, keyboards and pointers. and robot control.
He presents his company’s claims about data scale and a model operating across games and robot types, while noting that its demonstrations were private at the time. He also discusses simulated environments, accessibility and the distinction between action-labeled training and JEPA-style approaches.
Watch on YouTube
