#302 Karl Friston: How the Free Energy Principle Could Rewrite AI

We change our mind hundred milliseconds by hundred milliseconds as we engage with the world and we continue making sense, doing our sense making with the world. And that's a process of inference. It's not necessarily learning, which would be a much slower process. And if the way that our brains work can be cast as inference, that means if our brains don't work properly, as in mental disorders, then we can understand that as false inference. The brain is in the game of basically trying to predict what it's sensing. So if this is a prediction error, that's newsworthy. So it can now use the prediction error to update, to revise its explanation, its representations of the latent causes. So in the moment, I literally change my mind. I literally change my mind on the basis of the prediction errors that have been elicited by comparing what I predict and what I actually saw. Hi. The transformer algorithm created by a team at Google in 2017 has been the core of generative AI. But scaling transformers has hit practical theoretical limits. New releases are providing only incremental improvement, and as a result, many researchers are looking into new architectures aiming for better memory use, better generalization, and longer context reasoning. Among those researchers is the team at Verses, a startup implementing the ideas of Carl Friston, who is the most highly cited neuroscientist globally and one of the most cited living scientists overall. Tristan is the creator of the free energy principle, a sweeping attempt to explain how all living systems, including brains, maintain order in a chaotic world. His work has inspired entire branches of computational neuroscience and new directions in artificial intelligence. Versus, where he serves as chief scientist, was started by some of his students. Its core project is called Axiom, a groundbreaking AI architecture built on the principles of Friston's work. Taking inspiration from how the human brain predicts and adapts to its environment, not by memorizing data like today's models, but by constantly minimizing what Friston called surprise, which is a measure of the difference between prediction and reality. It's an ambitious attempt to create machines that learn and reason the way living systems do, grounding intelligence in physics rather than statistics. In fact, this is my second conversation with Friston because I was completely lost the first time around, and he says himself that he has trouble speaking in the vernacular. So while some listeners will be able to understand what he's saying, most are better off letting the jargon wash over them, focusing instead on the higher concepts and how they're implemented in an AI model. You may not understand everything, but you'll come away with a picture of how and why post transformer architectures are being developed. I hope you find the conversation as fascinating as I did. But first, I wanna give a shout out to our sponsor. Build the future of multi agent software with Agency. That's a g …