What are the key takeaways from this Practical AI episode?

Key insights include: **End-to-End Training Architecture:** Comma AI trains models directly from hundreds of millions of miles of human driving data, skipping intermediate detection layers like lane-line segmentation or traffic-light classifiers entirely. The model takes raw camera video as input and outputs two values: longitudinal acceleration and road curvature. This minimal output design keeps the on-device model small enough to run on a phone-grade chip.; **Diffusion Simulator as Training Environment:** Rather than classical depth-reprojection simulators, Comma AI now trains its driving policy inside a machine-learning-generated video simulator built on diffusion models. The critical differentiator is input-response accuracy — if the simulator is told the car turns left 10 degrees, it must produce video that precisely reflects that turn, not just photorealistic footage, making it viable for robotics training.; **Compute Gap and Its Practical Ceiling:** Comma AI's current device runs roughly 100 times less compute than a Tesla FSD computer. Despite this gap, highway performance is comparable because capability gains require exponential compute increases for marginal real-world improvements. A planned external GPU add-on targeting 100x more compute is projected to roughly double detection reliability in nuanced situations like ambiguous traffic lights.

What did Harold Schafer discuss on Practical AI?

Harald Sch, CTO at Comma AI, explains how OpenPilot — the most popular open source robotics project on GitHub — uses end-to-end machine learning and a diffusion-based world model simulator to deliver highway autonomy across supported vehicles, while outlining three unsolved problems blocking full autonomous driving: controls, reinforcement learning, and continual learning. Key topics include: **End-to-End Training Architecture:** Comma AI trains models directly from hundreds of millions of miles of human driving data, skipping intermediate detection layers like lane-line segmentation or traffic-light classifiers entirely. The model takes raw camera video as input and outputs two values: longitudinal acceleration and road curvature. This minimal output design keeps the on-device model small enough to run on a phone-grade chip.; **Diffusion Simulator as Training Environment:** Rather than classical depth-reprojection simulators, Comma AI now trains its driving policy inside a machine-learning-generated video simulator built on diffusion models. The critical differentiator is input-response accuracy — if the simulator is told the car turns left 10 degrees, it must produce video that precisely reflects that turn, not just photorealistic footage, making it viable for robotics training..

How long is this episode of Practical AI?

This episode is 46 minutes long. SignalCast provides an AI-generated summary so you can get the key insights in about 3 minutes.

Practical AI

Open Source Self-Driving with Comma AI

April 16, 2026

46 min episode · 2 min read

Harold Schafer

Episode

46 min

Read time

2 min

Topics

Fundraising & VC, Design & UX, Artificial Intelligence

AI-Generated Summary

Published Apr 16, 2026

Key Takeaways

✓End-to-End Training Architecture: Comma AI trains models directly from hundreds of millions of miles of human driving data, skipping intermediate detection layers like lane-line segmentation or traffic-light classifiers entirely. The model takes raw camera video as input and outputs two values: longitudinal acceleration and road curvature. This minimal output design keeps the on-device model small enough to run on a phone-grade chip.
✓Diffusion Simulator as Training Environment: Rather than classical depth-reprojection simulators, Comma AI now trains its driving policy inside a machine-learning-generated video simulator built on diffusion models. The critical differentiator is input-response accuracy — if the simulator is told the car turns left 10 degrees, it must produce video that precisely reflects that turn, not just photorealistic footage, making it viable for robotics training.
✓Compute Gap and Its Practical Ceiling: Comma AI's current device runs roughly 100 times less compute than a Tesla FSD computer. Despite this gap, highway performance is comparable because capability gains require exponential compute increases for marginal real-world improvements. A planned external GPU add-on targeting 100x more compute is projected to roughly double detection reliability in nuanced situations like ambiguous traffic lights.
✓Continual Learning as an Unsolved Requirement: OpenPilot currently uses classical optimization to learn vehicle-specific parameters — tire stiffness, friction coefficients — live during each drive. Inflating tires or driving in rain changes vehicle dynamics that the system must adapt to in real time. Standard neural network approaches cannot yet handle this live adaptation, making continual learning one of three critical unsolved problems for production autonomy.
✓Open Source as a Functional Requirement, Not Just Philosophy: Supporting hundreds of car models requires community contributors to reverse-engineer each vehicle's CAN bus signals. A closed-source stack would make this ecosystem impossible to scale. Comma AI treats open sourcing the car-interface layer as a structural necessity, while also holding a philosophical position that device owners should have full visibility into and control over software running on hardware they purchase.

What It Covers

Harald Sch, CTO at Comma AI, explains how OpenPilot — the most popular open source robotics project on GitHub — uses end-to-end machine learning and a diffusion-based world model simulator to deliver highway autonomy across supported vehicles, while outlining three unsolved problems blocking full autonomous driving: controls, reinforcement learning, and continual learning.

Key Questions Answered

•End-to-End Training Architecture: Comma AI trains models directly from hundreds of millions of miles of human driving data, skipping intermediate detection layers like lane-line segmentation or traffic-light classifiers entirely. The model takes raw camera video as input and outputs two values: longitudinal acceleration and road curvature. This minimal output design keeps the on-device model small enough to run on a phone-grade chip.
•Diffusion Simulator as Training Environment: Rather than classical depth-reprojection simulators, Comma AI now trains its driving policy inside a machine-learning-generated video simulator built on diffusion models. The critical differentiator is input-response accuracy — if the simulator is told the car turns left 10 degrees, it must produce video that precisely reflects that turn, not just photorealistic footage, making it viable for robotics training.
•Compute Gap and Its Practical Ceiling: Comma AI's current device runs roughly 100 times less compute than a Tesla FSD computer. Despite this gap, highway performance is comparable because capability gains require exponential compute increases for marginal real-world improvements. A planned external GPU add-on targeting 100x more compute is projected to roughly double detection reliability in nuanced situations like ambiguous traffic lights.
•Continual Learning as an Unsolved Requirement: OpenPilot currently uses classical optimization to learn vehicle-specific parameters — tire stiffness, friction coefficients — live during each drive. Inflating tires or driving in rain changes vehicle dynamics that the system must adapt to in real time. Standard neural network approaches cannot yet handle this live adaptation, making continual learning one of three critical unsolved problems for production autonomy.
•Open Source as a Functional Requirement, Not Just Philosophy: Supporting hundreds of car models requires community contributors to reverse-engineer each vehicle's CAN bus signals. A closed-source stack would make this ecosystem impossible to scale. Comma AI treats open sourcing the car-interface layer as a structural necessity, while also holding a philosophical position that device owners should have full visibility into and control over software running on hardware they purchase.

Notable Moment

Harald Sch referenced the early Waymo TED talk where a founder predicted his young children would never need driver's licenses — those children now have licenses. He used this to frame how far autonomous driving has come while calibrating realistic expectations about how far it still needs to go.

Know someone who'd find this useful?

Episode Transcript

Welcome to the Practical AI podcast, where we break down the real world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, x, or Blue Sky to stay up to date with episode drops, behind the scenes content, and AI insights. You can learn more at practicalai.fm. Now onto the show. Welcome Welcome to another episode of the Practical AI podcast. This is Daniel Whitenack. I'm CEO at Prediction Guard, and I'm joined as always by my co host, Chris Benson, who is a principal AI and autonomy research engineer. How How are you doing, Chris? Hey. Doing very well today. How's it going? It's it's going great. I was commenting to our guests today just before we started recording that earlier, this year, I was in a in the car with one of our one of our engineers, shout out to Ed, and he's like, hey, have you heard about this cool thing? We're driving in the car. Right? And he's like, hey, there's this cool thing you can put in your car and make it like a like a AI AI assisted driving car without it being, like, a specific self driving car. And, and so he he, forwarded me the information about Kama, and I'm really excited today to welcome Harold Schafer, who is CTO at Kama AI. Welcome, Harold. Thank you. Thank you for having me. Excited to be here. Yeah. Yeah. Well, obviously, I I kind of alluded to some of what you're involved with, but maybe could you give us just a little bit of background, about yourself and Kama and kind of how you ended up in this spot of working on, working on some of the things that you're working on. Sure. So Kama makes this device, like you said, that you can install in cars and gives them autonomy features that they hadn't had before, you know, things like auto steer and and better, ACC. So on on the highway, you kind of get some level of autonomy. And, you know, the software that runs that is is OpenPilot, and that's a completely open source, autonomy stack for cars. By far the most popular open source, self driving stack online. And, I think it's it's currently even the most popular robotics project, on on GitHub. So that's kinda where we're at. I've been working on this a really long time. I've been at Karma for, nine years now. So basically, been my entire life. I don't think there's there there's much to say about my professional life that's not related to COMMA. But I came to The US, like, ten years ago, graduated, and started working here. And so been working on OpenPilot …

Get the full transcript (8,906 words) + summary by email — free

One-time email with the complete transcript and AI summary of this episode. No account needed.

One email, no spam. We’ll also show you what SignalCast does.

Browse all Practical AI transcripts →

You just read a 3-minute summary of a 43-minute episode.

Get Practical AI summarized like this every Monday — plus up to 2 more podcasts, free.

Pick Your Podcasts — Free

Books, tools, and gear mentioned in this episode

SignalCast may earn commission on purchases via these links.

Tools

OpenPilotBy guest
by Comma AI
“Harald Sch, CTO at Comma AI, explains how OpenPilot — the most popular open source robotics project on GitHub — uses end-to-end machine learning and a diffusion-based world model simulator to deliver highway autonomy across supported vehicles”