Baseten CEO Tuhin Srivastava on the AI Inference Crunch, Custom Models, and Building the Inference Cloud

Hi, listeners. Today, Elad and I are here with Tuhen Srivastava, the founder and CEO of Base ten, the AI Inference Cloud. We're here to talk about capacity constraints for AI capacity constraints for AI compute, why inference is the last market, how the workload is changing, the open source, and perhaps multi chip future, and what 30 x scale in a year looks like. Tuhen, welcome back. Hi. Good to see you. Thanks for having me. Alright. You are in one of the, craziest markets, AI inference. It's very important. There's a lot going on. You guys have grown 30 x over the last year. And I think I can say you're expecting to do more than a billion dollars in revenue this year. Mhmm. What's going on? Tell us about scale. Yeah. No. It's been it's been nuts. I I think what's happened over the last, honestly, at twenty four months, but just kinda keeps getting bigger and bigger is that I think everyone is real realizing that you can put AI everywhere. You have you have all these great options available from closed source to open source models. The open source models have crossed some sort of chasm in terms of their baseline capability, and then I think RL techniques and post training is, for specialized models, has become mainstream enough, and, you know, there's enough examples of it work of it working. The customers are realizing they can, you know, kind of own their inference, more and more. And what that's meant for us is, more, you know, the long tail models coming through, customers in housing a lot of that intelligence themselves. And as the application layer just gets, you know, bigger and bigger and bigger and that's growing, we are we are just someone index on that, and we've been around to be able to collect the demand. There's an existential question in here that I think everybody is, continually asking of, does the independent application layer get to exist at all versus the labs? Like, how do you you have to believe this. Why do you believe it? Yeah. Look. I I I think it'd be it'd be a sad thing if it didn't exist in general, and I think that's, like, my but, you know, sadness is fine. The Sadness is fine. But, like, that that that's not the reason why I think the application layer will exist. I think the application layer will exist for a number of reasons. One is because, you know, I think this idea, that what it what is valuable to a company, is, you know, the the user signal that they can gather, that only they can gather. And to the extent that that is encoded, in a model, I think a lot of their business will, be at risk. But to the to the extent that it is encoded in workflows, that is where they will be able to develop notes. So a good I …