Ming-Yu Liu describes an architecture that connects a vision-language model with a bidirectional generator for video, action and audio. Tim Scarfe asks how modalities with different frequencies can work together; temporal position embeddings place their tokens on a shared time axis.
Ming-Yu Liu treats forward dynamics, inverse dynamics and policy as related ways to learn the connection between observation and action. Shared capacity can encourage transfer across tasks and embodiments, but he acknowledges that the model does not itself reliably know when transfer will be harmful.
Tim Scarfe raises ambiguous instructions and safety-critical deployment. Ming-Yu Liu describes system-level planning, step-by-step execution and checks around the model, rather than claiming a world model alone resolves those requirements.
Ming-Yu Liu proposes passive policy verification as an early use for neural simulators: simulated rankings could narrow the checkpoints that need expensive physical tests if those rankings correlate with real-world results. He acknowledges simulator exploitation and does not present a settled method that eliminates it.
Ming-Yu Liu distinguishes relatively tractable navigation from manipulation involving contact and deformation. The discussion also considers local inference when network latency or connectivity is unacceptable. These are the guest's explanations and research expectations, not proof that a simulated policy is safe to deploy.
Watch on YouTube
