NVIDIA’s Marco Pavone on AI Simulation, Safety, and the Road to Autonomous Vehicles - Ep. 260

Hello, and welcome to the NVIDIA AI podcast. I'm your host, Noah Kravitz. If you've been in San Francisco, Phoenix, or a number of other cities recently, no doubt you've seen cars navigating the streets without anyone in the driver's seat. Robotaxis and other autonomous vehicles are becoming a part of our lives, whether through prototype testing on public roads or driverless cars ferrying passengers across city streets. Autonomous vehicles hold a lot of promise, but first and foremost, they have to be safe. This is where advanced simulations powered by AI can help. Here to explain the ins and outs of autonomous vehicle safety and simulation and why it's so important to all of us going forward is Marco Pavone, senior director of autonomous vehicle research at NVIDIA and professor at Stanford University. Marco, welcome, and thank you so much for taking the time to join the NVIDIA AI podcast. Thank you for having me. So, Marco, if you don't mind, could we start by having you tell us a little bit about what your role is at NVIDIA and and Stanford as well, if applicable, when it comes to autonomous vehicle research, safety, simulation, and everything else we're gonna talk about. Sure. At, NVIDIA, I lead the autonomous vehicle, research. And, our research spans, a number of, different research tasks. First of all, we work on developing new architectures for autonomous, systems. We work on, developing a d foundation models that can empower the full development life cycle, the full development program from the onboard stack all the way to simulation, data curation, and even safety evaluation. We work on, enabling a high fidelity behavior and sensor simulation. And we also work on developing tools to ensure the safety of AI based, stocks. Right. At NVIDIA, I focus primarily on ground robotic systems, namely self driving cars, while at Stanford, I focus primarily on autonomous aerospace systems. How fascinating. We'll we'll have to have you back to do an episode on the the potential future of flying cars, putting those two things together. So in the introduction, I mentioned, robotaxis. It was something that came to mind for me as an easy way to kind of visualize what we're talking about. But as I understand it, your purview and what NVIDIA is working on more broadly isn't just fully self driving vehicles. There's more to it than that. Yes. The field of, autonomous vehicle is very diverse, both in terms of the technology from, semi automated systems all the way to fully automated systems, like, for example, the robotaxis. And also in terms of application domains, personal mobility, namely, self driving cars for people, you know, moving in cities, all the way to freight transportation, autonomous vehicles for agriculture, for construction, and, more. Right. Excellent. So let's start maybe by talking about halos. NVIDIA recently announced well, I'm gonna leave it to you, something called halos. Can you can you tell us about it and how it relates …