One Brain, Any Robot: Skild AI's Skild Brain Explained - Ep. 295
Episode
29 min
Read time
2 min
Topics
Startups, Fundraising & VC, Design & UX
AI-Generated Summary
Key Takeaways
- ✓Three-Source Data Architecture: Skild trains OmniBrain using video data (billions of examples, high diversity, low precision), simulation data (scalable, measurable forces, but sim-to-real gap exists), and real-world teleoperation data (richest quality, hardest to scale). Each source compensates for the others' weaknesses, mirroring the pre-training and fine-tuning separation used in large language models.
- ✓Deployment as a Technical Problem: Unlike software products where users self-onboard, robotics deployment requires active engineering effort. Skild's strategy is to get new robot systems operational within days using minimal fine-tuning data, then scale across scenarios. This rapid specialization from a general base model is treated as a core technical challenge, not an afterthought.
- ✓Data Flywheel Across Verticals: Corner cases in one industry become standard training cases for another. Structured factory deployments generate data that enables semi-structured environments like hospitals and hotels, which in turn bootstrap unstructured home robotics. This cross-vertical data accumulation is the primary mechanism Skild uses to improve OmniBrain over time.
- ✓Three-Stage Deployment Testing Protocol: Before any deployment, Skild runs task-specific KPIs (accuracy and cycle time), generalization stress tests (altered lighting, unexpected objects, hardware failures), and safety guardrail validation. If a camera feed is severed, for example, the robot must halt or stay within predefined boundaries rather than continue operating blind.
- ✓NVIDIA Stack Integration: Skild uses Isaac physics simulation for scenario generation, Cosmos generative models for data augmentation to create variations from single data points, and NVIDIA edge compute hardware for on-device inference. On-device processing is necessary because robots cannot tolerate server round-trip latency during physical actions like catching or stabilizing.
What It Covers
Skild AI co-founders Deepak Pathak and Abhinav Gupta explain their OmniBrain platform — a single universal model designed to control any robot form factor across any task, using a three-source data strategy and deployment-first approach to scale physical AI across industrial and consumer environments.
Key Questions Answered
- •Three-Source Data Architecture: Skild trains OmniBrain using video data (billions of examples, high diversity, low precision), simulation data (scalable, measurable forces, but sim-to-real gap exists), and real-world teleoperation data (richest quality, hardest to scale). Each source compensates for the others' weaknesses, mirroring the pre-training and fine-tuning separation used in large language models.
- •Deployment as a Technical Problem: Unlike software products where users self-onboard, robotics deployment requires active engineering effort. Skild's strategy is to get new robot systems operational within days using minimal fine-tuning data, then scale across scenarios. This rapid specialization from a general base model is treated as a core technical challenge, not an afterthought.
- •Data Flywheel Across Verticals: Corner cases in one industry become standard training cases for another. Structured factory deployments generate data that enables semi-structured environments like hospitals and hotels, which in turn bootstrap unstructured home robotics. This cross-vertical data accumulation is the primary mechanism Skild uses to improve OmniBrain over time.
- •Three-Stage Deployment Testing Protocol: Before any deployment, Skild runs task-specific KPIs (accuracy and cycle time), generalization stress tests (altered lighting, unexpected objects, hardware failures), and safety guardrail validation. If a camera feed is severed, for example, the robot must halt or stay within predefined boundaries rather than continue operating blind.
- •NVIDIA Stack Integration: Skild uses Isaac physics simulation for scenario generation, Cosmos generative models for data augmentation to create variations from single data points, and NVIDIA edge compute hardware for on-device inference. On-device processing is necessary because robots cannot tolerate server round-trip latency during physical actions like catching or stabilizing.
Notable Moment
The founders use Roger Federer as a concrete illustration of video data's limits: watching millions of hours of professional tennis footage does not produce a professional tennis player. Observation alone cannot transfer physical skill — robots, like humans, require practice repetitions to build reliable motor behavior.
Episode Transcript
Robotics is a data problem. Like, unlike language or vision, there is not much data in robotics. There is no Internet of robot data. So if that's the scenario, we cannot pick and choose which data we use. So we go in a most general fashion every single instance of our brain which we deploy for any kind of task or any form factor that contributes in making the brain better for the future scenarios. Welcome to the NVIDIA AI podcast. I'm Noah Kravitz. I'm here today with Deepak Pathak and Avanav Gupta from SKILLD. SKILLD is a robotics company that's building the Omni Brain, a universal brain that can power robots across any form factor to tackle any task. It's amazing stuff. Very excited to find out about it from the source. And so let's get into it. Deepak, Avanav, welcome. Thank you so much for joining the AI podcast. Thank you so much for having us. So Deepak, maybe you can start and tell us a little bit about the company, about SKILLED, and then you can both talk a little bit about your roles. Yeah. So at SKILLED, as you mentioned, we are building a general purpose brain. So we call this only bodied intelligence. Any robot, any task, one brain. So think of like what GPT is for language. We are building a general brain for any physical device or any kind of robot. So this is absolutely general. Like, you can have a humanoid or a dog like robot or a robotic arm on a conveyor belt all being controlled by the same shared brain, shared intelligence behind the scene. So why do we go so general? And the reason is robotics is a data problem. Like unlike language or vision, there is not much data in robotics. There is no Internet of robot data. So if that's the scenario, we cannot pick and choose which data we use. So we go in a most general fashion every single instance of our brain which we deploy for any kind of task or any form factor that contributes in making the brain better for the future scenarios. So this is the main, goal, behind this. Yeah. And personally in my role, I have been we both have been professors before this so we are extremely technical. We have been involved in in bringing of these technologies, in the robot learning area for the last decade and more. So our role is both on the technical side to make sure that these things get built and they are super general, transferable. But our focus is also a lot on deployments. Right. Like we do not believe deployment to be a it's not hindsight, a scenario. Like for for for instance, in case of Charge GPD or or language models, folks did research for several years, but once it was ready, you have million users in seven, seven days. Only one day. I don't remember. Maybe 100,000,000 users …
Get the full transcript (5,774 words) + summary by email — free
One-time email with the complete transcript and AI summary of this episode. No account needed.
One email, no spam. We’ll also show you what SignalCast does.
You just read a 3-minute summary of a 26-minute episode.
Get NVIDIA AI Podcast summarized like this every Monday — plus up to 2 more podcasts, free.
Pick Your Podcasts — FreeKeep Reading
More from NVIDIA AI Podcast
Inside Instacart's AI-Powered Smart Shopping Cart | NVIDIA AI Podcast Ep. 302
Jun 24 · 39 min
No Priors: Artificial Intelligence | Technology | Startups
Rethinking Legacy Data Infrastructure with Eon Co-Founders Ofir Ehrlich and Gonen Stein
Aug 27
More from NVIDIA AI Podcast
How Mistral Is Building Frontier AI for the Enterprise | NVIDIA AI Podcast Ep. 301
Jun 10 · 21 min
a16z Podcast
How Global Networks Are Reshaping Startup Success
Aug 20
Books, tools, and gear mentioned in this episode
SignalCast may earn commission on purchases via these links. As an Amazon Associate, SignalCast earns from qualifying purchases.
Tools
by NVIDIA
“Skild uses Isaac physics simulation for scenario generation, Cosmos generative models for data augmentation to create variations from single data points, and NVIDIA edge compute hardware for on-device inference.”
by NVIDIA
“Skild uses Isaac physics simulation for scenario generation, Cosmos generative models for data augmentation to create variations from single data points, and NVIDIA edge compute hardware for on-device inference.”
Products
by Skild AI
“Skild AI co-founders Deepak Pathak and Abhinav Gupta explain their OmniBrain platform — a single universal model designed to control any robot form factor across any task, using a three-source data strategy and deployment-first approach to scale physical AI across industrial and consumer environments.”
More from NVIDIA AI Podcast
We summarize every new episode. Want them in your inbox?
Inside Instacart's AI-Powered Smart Shopping Cart | NVIDIA AI Podcast Ep. 302
How Mistral Is Building Frontier AI for the Enterprise | NVIDIA AI Podcast Ep. 301
Everyone Can Build a Robot: Open Source Embodied AI With Seeed Studio | NVIDIA AI Podcast Ep. 300
Inside AI Tokenomics: How to Profitably Turn Tokens Into Business Value | NVIDIA AI Podcast Ep. 299
Snap’s Secret to Processing 10 Petabytes a Day: GPU-Accelerated Spark | NVIDIA AI Podcast Ep. 298
Similar Episodes
Related episodes from other podcasts
No Priors: Artificial Intelligence | Technology | Startups
Aug 27
Rethinking Legacy Data Infrastructure with Eon Co-Founders Ofir Ehrlich and Gonen Stein
a16z Podcast
Aug 20
How Global Networks Are Reshaping Startup Success
This Week in Startups
Aug 19
Neurosymbolic AI outperforms chatbots and product search | E2327
David Senra
Aug 12
Lulu Cheng Meservey, Founder of Rostra
Latent Space
Aug 11
🔬The BioAI Phase Shift - Matthew McPartlon & Neil Patil, Chai Discovery
Explore Related Topics
This podcast is featured in Best AI Podcasts (2026) — ranked and reviewed with AI summaries.
Read this week's Startups & Product Podcast Insights — cross-podcast analysis updated weekly.
You're clearly into NVIDIA AI Podcast.
Every Monday, we deliver AI summaries of the latest episodes from NVIDIA AI Podcast and 192+ other podcasts. Free for one show.
Start My Monday DigestNo credit card · Unsubscribe anytime