Ming-Yu Liu

AI Summary

→ WHAT IT COVERS NVIDIA's Mingyu Liu explains world foundation models, deep learning systems that simulate physics and predict futures to train and verify physical AI agents like humanoid robots and autonomous vehicles through the new Cosmos development platform. → KEY INSIGHTS - **Three verification applications:** World models test policy checkpoints in simulation before physical deployment, initialize policy models with pretrained physics understanding to reduce training data needs, and enable real-time future rollout simulation for decision-making before robots act. - **Cosmos platform components:** NVIDIA releases open-weight models free for commercial use, including diffusion-based and autoregressive architectures, video tokenizers for transformer processing, post-training scripts for custom camera configurations, and video curation toolkits leveraging GPU-accelerated libraries for data processing. - **Autoregressive versus diffusion tradeoffs:** Autoregressive models predict tokens sequentially like GPT, run faster due to existing optimizations, but struggle with video compression accuracy. Diffusion models generate token sets simultaneously, produce more coherent outputs with better quality, but require different integration approaches. - **Early industry adoption focus:** Self-driving car companies and humanoid robot manufacturers including One X, Huobi, Li Auto, and others partner with NVIDIA to apply world models for testing scenarios difficult to replicate physically and validating agent behavior across diverse environments. → NOTABLE MOMENT Liu compares world models to having a strategic advisor simulate different futures before making decisions, allowing physical AI systems to avoid costly real-world mistakes by testing policies in thousands of virtual kitchens before deploying in actual environments. 💼 SPONSORS None detected 🏷️ World Foundation Models, Physical AI, Autonomous Vehicles, Humanoid Robotics