Robot Learning: Learning to Plan With Large Models

Watch a technical lecture exploring latent inverse models and their applications in video generation and robotics. Dive into how these models develop predictive capabilities by embedding world dynamics understanding, particularly for complex scenarios involving human motion. Learn about conditioning predictions on latent action models for controlled manipulation, and understand the critical role of action codebooks in task-specific applications. Explore the importance of data scale and image fidelity in capturing detailed dynamics, with special focus on the "Genie" project's findings. Gain insights into the training process of latent dynamics models and their ability to infer underlying states from image data, while understanding how this approach differs from traditional inverse dynamics models through its integrated action control capabilities.