AI, Spatial Intelligence, and 3D Content Creation With Sanja Fidler of NVIDIA - Ep. 269

What It Covers

Sanja Fidler, VP of AI Research at NVIDIA, explains spatial intelligence and physical AI development, covering how her Toronto lab creates three-dimensional world models, simulation platforms, and robotics training environments through Omniverse.

Key Questions Answered

• •Simulation-Based Robot Training: Physical AI requires virtual playgrounds that accurately mimic real-world physics to train robots safely and cost-effectively, avoiding expensive real-world trial and error that would damage equipment and environments during the learning process.
• •Differentiable Rendering Pipeline: NVIDIA doubled down on making graphics pipelines compatible with AI by lifting images and videos to three-dimensional representations, enabling phone-based room scanning that instantly creates training environments in Isaac simulator for immediate robot deployment.
• •World Models Evolution: Video-based world models learn physics from real-world recordings without human editing, progressing from GAN-based systems like GameGAN and DriveGAN to latent diffusion models that now power physically accurate simulations at scale through platforms like Cosmos.
• •Visual Language Model Integration: VLMs represent the breakthrough for handling long-tail scenarios in robotics by bringing language-based reasoning into physical environments, allowing systems to navigate completely new situations never encountered during training through semantic understanding and reasoning capabilities.