Dataflow Computing for AI Inference with Kunle Olukotun - #751

We have a data flow problem, right? So let's build a data flow computer, right? So let's build a computer that matches the data flows as represented by the algorithm in the hardware. So why does it need to be reconfigurable? Well, because you don't have one graph. Right? You it's your it's the it's how you change what the what the hardware does. You reconfigure it. Alright, everyone. Welcome to another episode of the Twindler AI podcast. I am your host, Sam Charrington. Today, I'm joined by Kunle Olukotun. Kunle is a professor of electrical engineering and computer science at Stanford University and cofounder and chief technologist at SambaNova Systems. Before we get going, be sure to take a moment to hit that subscribe button wherever you're listening to today's show. Kunle, welcome back to the podcast. It has been a minute. Good to be back and, glad to be here. We were just chatting. It was, I think at a, when we first spoke back in 2018. Clearly a very different world in AI since then, and it is going to be really good to catch up with you and, what's been new in your research and at SambaNova. Let's get started by having you share a little bit about your background and what you've been working on recently. Well, thank you for having me on again, and it's, good to be back on on the, the the cast podcast. And, you know, my background is is, you know, is computer architect, well known, for doing some of the, pioneering multicore work, back in the in the mid nineties, and, done a lot of work on parallel programming environments using domain specific languages. And, this work has has led to to the, work we're doing now in research and, at San Benova about how to take, reconfigurable data flow architectures and, efficiently execute, these domain specific language models, to create environments that are better for doing AI computation. Right? And so, our focus is on how you, do very efficient fast inference on big models, as energy efficiently as possible. Right? And so, the focus has been on on very large models, you know, models with, trillions of parameters that can fit in a single rack, on models that, are good for agentic solutions, where you've got hundreds of agents that, potentially, need to, work together and and switch between these agents with very low latency, and then sort of very fast inference because you, have interactive environments that you want to support, multiple LM model calls, sequentially for a chain of thought, kinds of, reasoning or because you've got a workflow that requires lots of all end calls. Yeah. You mentioned in there the idea of reconfigurable data flow architecture. Let's kinda step back for a second and talk a little bit about, you know, what that means. What is a a reconfigurable data flow architecture, and where does that idea come from? Well, it comes basically …