Modal and Scaling AI Inference with Erik Bernhardsson

Modal is a serverless compute platform that's specifically focused on AI workloads. The company's goal is to enable AI teams to quickly spin up GPU enabled containers and rapidly iterate and auto scale. It was founded by Eric Bernardson, who was previously at Spotify for seven years where he built the music recommendation system and the popular Luigi workflow scheduler. In this episode, Eric joins Sean Falconer to talk about the motivation for founding his company, the market gap in ML and AI tooling, optimizing container cold start, Modal's interface design, and more. This episode is hosted by Sean Falconer. Check the show notes for more information on Sean's work and where to find him. Eric, welcome to the show. Thank you. It's great to be here. Yeah. Thanks so much for being here. So I was diving a little bit into your background preparing for this. And so it seems like you spent a lot of your time working in data throughout your career, which also kinda matches my own experience. You know, you previously were at Spotify for a number of years. You were the CTO of better.com. Now you're the founder and CEO of Modal. You know, were there certain things in these prior roles that led to identifying some sort of need for Modal? Like, what's the story behind essentially going off and deciding to start this company? Yeah. For sure. The answer is yes. And the long story is I was at Spotify for seven years, built a music recommendation system. But as a part of building that, I also realized there's kind of a general gap in the tooling. I ended up building a vector database called Illinois. I know we use it today. And also workflow scheduler called Luigi that very few people use today. But generally, I realized, like, as a part of building all of that stuff at Spotify and also did a lot of other stuff that there's very little tooling in data, AI, machine learning. There's more today, but it still never really felt like much later, like, in 2020, 2021 when I started thinking about building a company. I realized there's kind of a gap in the market for, like, a tool I always wanted to have myself. So that was kind of the genesis of modal. It's almost like building selfishly for what I always wanted to have, you know, throughout my years as recognition system, and also, to some extent, it better as the CTO. So a little bit more like general role. We spent a lot of time thinking about platforms and data and stuff like that too. Yeah. I mean, sometimes people talk about how, you know, the discipline is essentially, like or, like, the tooling and the things that are available for data engineers lag somewhere, you know, five years maybe behind, you know, more app traditional application development. Where would you say that something like ML engineering from an applied sense of …