Skip to main content
NVIDIA AI Podcast

One Brain, Any Robot: Skild AI's Skild Brain Explained - Ep. 295

29 min episode · 2 min read
·
Deepak Pathak,Avanav Gupta

Episode

29 min

Read time

2 min

Topics

Startups, Fundraising & VC, Design & UX

AI-Generated Summary

Key Takeaways

  • Three-Source Data Architecture: Skild trains OmniBrain using video data (billions of examples, high diversity, low precision), simulation data (scalable, measurable forces, but sim-to-real gap exists), and real-world teleoperation data (richest quality, hardest to scale). Each source compensates for the others' weaknesses, mirroring the pre-training and fine-tuning separation used in large language models.
  • Deployment as a Technical Problem: Unlike software products where users self-onboard, robotics deployment requires active engineering effort. Skild's strategy is to get new robot systems operational within days using minimal fine-tuning data, then scale across scenarios. This rapid specialization from a general base model is treated as a core technical challenge, not an afterthought.
  • Data Flywheel Across Verticals: Corner cases in one industry become standard training cases for another. Structured factory deployments generate data that enables semi-structured environments like hospitals and hotels, which in turn bootstrap unstructured home robotics. This cross-vertical data accumulation is the primary mechanism Skild uses to improve OmniBrain over time.
  • Three-Stage Deployment Testing Protocol: Before any deployment, Skild runs task-specific KPIs (accuracy and cycle time), generalization stress tests (altered lighting, unexpected objects, hardware failures), and safety guardrail validation. If a camera feed is severed, for example, the robot must halt or stay within predefined boundaries rather than continue operating blind.
  • NVIDIA Stack Integration: Skild uses Isaac physics simulation for scenario generation, Cosmos generative models for data augmentation to create variations from single data points, and NVIDIA edge compute hardware for on-device inference. On-device processing is necessary because robots cannot tolerate server round-trip latency during physical actions like catching or stabilizing.

What It Covers

Skild AI co-founders Deepak Pathak and Abhinav Gupta explain their OmniBrain platform — a single universal model designed to control any robot form factor across any task, using a three-source data strategy and deployment-first approach to scale physical AI across industrial and consumer environments.

Key Questions Answered

  • Three-Source Data Architecture: Skild trains OmniBrain using video data (billions of examples, high diversity, low precision), simulation data (scalable, measurable forces, but sim-to-real gap exists), and real-world teleoperation data (richest quality, hardest to scale). Each source compensates for the others' weaknesses, mirroring the pre-training and fine-tuning separation used in large language models.
  • Deployment as a Technical Problem: Unlike software products where users self-onboard, robotics deployment requires active engineering effort. Skild's strategy is to get new robot systems operational within days using minimal fine-tuning data, then scale across scenarios. This rapid specialization from a general base model is treated as a core technical challenge, not an afterthought.
  • Data Flywheel Across Verticals: Corner cases in one industry become standard training cases for another. Structured factory deployments generate data that enables semi-structured environments like hospitals and hotels, which in turn bootstrap unstructured home robotics. This cross-vertical data accumulation is the primary mechanism Skild uses to improve OmniBrain over time.
  • Three-Stage Deployment Testing Protocol: Before any deployment, Skild runs task-specific KPIs (accuracy and cycle time), generalization stress tests (altered lighting, unexpected objects, hardware failures), and safety guardrail validation. If a camera feed is severed, for example, the robot must halt or stay within predefined boundaries rather than continue operating blind.
  • NVIDIA Stack Integration: Skild uses Isaac physics simulation for scenario generation, Cosmos generative models for data augmentation to create variations from single data points, and NVIDIA edge compute hardware for on-device inference. On-device processing is necessary because robots cannot tolerate server round-trip latency during physical actions like catching or stabilizing.

Notable Moment

The founders use Roger Federer as a concrete illustration of video data's limits: watching millions of hours of professional tennis footage does not produce a professional tennis player. Observation alone cannot transfer physical skill — robots, like humans, require practice repetitions to build reliable motor behavior.

Know someone who'd find this useful?

Episode Transcript

Robotics is a data problem. Like, unlike language or vision, there is not much data in robotics. There is no Internet of robot data. So if that's the scenario, we cannot pick and choose which data we use. So we go in a most general fashion every single instance of our brain which we deploy for any kind of task or any form factor that contributes in making the brain better for the future scenarios. Welcome to the NVIDIA AI podcast. I'm Noah Kravitz. I'm here today with Deepak Pathak and Avanav Gupta from SKILLD. SKILLD is a robotics company that's building the Omni Brain, a universal brain that can power robots across any form factor to tackle any task. It's amazing stuff. Very excited to find out about it from the source. And so let's get into it. Deepak, Avanav, welcome. Thank you so much for joining the AI podcast. Thank you so much for having us. So Deepak, maybe you can start and tell us a little bit about the company, about SKILLED, and then you can both talk a little bit about your roles. Yeah. So at SKILLED, as you mentioned, we are building a general purpose brain. So we call this only bodied intelligence. Any robot, any task, one brain. So think of like what GPT is for language. We are building a general brain for any physical device or any kind of robot. So this is absolutely general. Like, you can have a humanoid or a dog like robot or a robotic arm on a conveyor belt all being controlled by the same shared brain, shared intelligence behind the scene. So why do we go so general? And the reason is robotics is a data problem. Like unlike language or vision, there is not much data in robotics. There is no Internet of robot data. So if that's the scenario, we cannot pick and choose which data we use. So we go in a most general fashion every single instance of our brain which we deploy for any kind of task or any form factor that contributes in making the brain better for the future scenarios. So this is the main, goal, behind this. Yeah. And personally in my role, I have been we both have been professors before this so we are extremely technical. We have been involved in in bringing of these technologies, in the robot learning area for the last decade and more. So our role is both on the technical side to make sure that these things get built and they are super general, transferable. But our focus is also a lot on deployments. Right. Like we do not believe deployment to be a it's not hindsight, a scenario. Like for for for instance, in case of Charge GPD or or language models, folks did research for several years, but once it was ready, you have million users in seven, seven days. Only one day. I don't remember. Maybe 100,000,000 users …

Get the full transcript (5,774 words) + summary by email — free

One-time email with the complete transcript and AI summary of this episode. No account needed.

One email, no spam. We’ll also show you what SignalCast does.

Browse all NVIDIA AI Podcast transcripts →

You just read a 3-minute summary of a 26-minute episode.

Get NVIDIA AI Podcast summarized like this every Monday — plus up to 2 more podcasts, free.

Pick Your Podcasts — Free

Keep Reading

Books, tools, and gear mentioned in this episode

SignalCast may earn commission on purchases via these links. As an Amazon Associate, SignalCast earns from qualifying purchases.

Tools

  • by NVIDIA

    Skild uses Isaac physics simulation for scenario generation, Cosmos generative models for data augmentation to create variations from single data points, and NVIDIA edge compute hardware for on-device inference.
  • by NVIDIA

    Skild uses Isaac physics simulation for scenario generation, Cosmos generative models for data augmentation to create variations from single data points, and NVIDIA edge compute hardware for on-device inference.

Products

  • by Skild AI

    Skild AI co-founders Deepak Pathak and Abhinav Gupta explain their OmniBrain platform — a single universal model designed to control any robot form factor across any task, using a three-source data strategy and deployment-first approach to scale physical AI across industrial and consumer environments.

More from NVIDIA AI Podcast

We summarize every new episode. Want them in your inbox?

Similar Episodes

Related episodes from other podcasts

Explore Related Topics

This podcast is featured in Best AI Podcasts (2026) — ranked and reviewed with AI summaries.

Read this week's Startups & Product Podcast Insights — cross-podcast analysis updated weekly.

You're clearly into NVIDIA AI Podcast.

Every Monday, we deliver AI summaries of the latest episodes from NVIDIA AI Podcast and 192+ other podcasts. Free for one show.

Start My Monday Digest

No credit card · Unsubscribe anytime