What are the key takeaways from this Eye on AI episode?

Key insights include: **Hebbian Plasticity over Fixed Weights:** Networks trained with local Hebbian learning rules — where connection strength changes based on how often paired neurons fire together — demonstrate real-time adaptability that static networks lack. A quadrupedal robot controlled by such a network continues functioning after losing a leg, despite never encountering that scenario during training, because weights continuously update throughout operation.; **Evolutionary Model Merging:** Rather than training new models from scratch, evolution can identify which layers from existing pretrained models to combine. Sakana AI demonstrated this by merging a Japanese-language model with a math-specialized model, producing a single model competent in both domains — a scalable strategy for capability expansion without full retraining cycles.; **LLMs as Mutation Operators:** Evolutionary search becomes significantly more powerful when a language model replaces hand-coded mutation functions. In circle-packing optimization, an LLM generates solution variants, fitness scores rank them, and the process iterates — navigating solution spaces that gradient descent cannot traverse because no differentiable objective exists across discrete or code-based representations.

What did Sebastian Risi discuss on Eye on AI?

Sebastian Risi, researcher at Sakana AI, explains neuroevolution — using evolutionary algorithms instead of gradient descent to optimize neural networks — and explores biologically inspired approaches including plastic networks, growing architectures, and combining large language models with evolutionary search to advance AI capabilities. Key topics include: **Hebbian Plasticity over Fixed Weights:** Networks trained with local Hebbian learning rules — where connection strength changes based on how often paired neurons fire together — demonstrate real-time adaptability that static networks lack. A quadrupedal robot controlled by such a network continues functioning after losing a leg, despite never encountering that scenario during training, because weights continuously update throughout operation.; **Evolutionary Model Merging:** Rather than training new models from scratch, evolution can identify which layers from existing pretrained models to combine. Sakana AI demonstrated this by merging a Japanese-language model with a math-specialized model, producing a single model competent in both domains — a scalable strategy for capability expansion without full retraining cycles..

How long is this episode of Eye on AI?

This episode is 59 minutes long. SignalCast provides an AI-generated summary so you can get the key insights in about 3 minutes.

Eye on AI

#330 Sebastian Risi: Why AI Should Be Grown, Not Trained

April 2, 2026

59 min episode · 2 min read

Sebastian Risi

Episode

59 min

Read time

2 min

Topics

Health & Wellness, Artificial Intelligence, Software Development

AI-Generated Summary

Published Apr 2, 2026

Key Takeaways

✓Hebbian Plasticity over Fixed Weights: Networks trained with local Hebbian learning rules — where connection strength changes based on how often paired neurons fire together — demonstrate real-time adaptability that static networks lack. A quadrupedal robot controlled by such a network continues functioning after losing a leg, despite never encountering that scenario during training, because weights continuously update throughout operation.
✓Evolutionary Model Merging: Rather than training new models from scratch, evolution can identify which layers from existing pretrained models to combine. Sakana AI demonstrated this by merging a Japanese-language model with a math-specialized model, producing a single model competent in both domains — a scalable strategy for capability expansion without full retraining cycles.
✓LLMs as Mutation Operators: Evolutionary search becomes significantly more powerful when a language model replaces hand-coded mutation functions. In circle-packing optimization, an LLM generates solution variants, fitness scores rank them, and the process iterates — navigating solution spaces that gradient descent cannot traverse because no differentiable objective exists across discrete or code-based representations.
✓Quality Diversity over Single-Objective Optimization: Optimizing purely for fitness score causes evolutionary systems to get trapped — a network learning a T-maze reward task performs worse than random chance by consistently choosing the smaller reward. Researchers should apply quality diversity algorithms that simultaneously reward exploration breadth and solution quality, preventing premature convergence to locally decent but globally poor strategies.
✓Co-evolving Agent and Environment: Training agents against static environments produces brittle specialists. The POET algorithm and its successors evolve terrain difficulty alongside the agent, starting simple and progressively increasing complexity. This curriculum approach enables bipedal robots to eventually navigate obstacle courses they could never learn directly — a principle now extendable using LLMs to generate Unity environments via code.

What It Covers

Sebastian Risi, researcher at Sakana AI, explains neuroevolution — using evolutionary algorithms instead of gradient descent to optimize neural networks — and explores biologically inspired approaches including plastic networks, growing architectures, and combining large language models with evolutionary search to advance AI capabilities.

Key Questions Answered

•Hebbian Plasticity over Fixed Weights: Networks trained with local Hebbian learning rules — where connection strength changes based on how often paired neurons fire together — demonstrate real-time adaptability that static networks lack. A quadrupedal robot controlled by such a network continues functioning after losing a leg, despite never encountering that scenario during training, because weights continuously update throughout operation.
•Evolutionary Model Merging: Rather than training new models from scratch, evolution can identify which layers from existing pretrained models to combine. Sakana AI demonstrated this by merging a Japanese-language model with a math-specialized model, producing a single model competent in both domains — a scalable strategy for capability expansion without full retraining cycles.
•LLMs as Mutation Operators: Evolutionary search becomes significantly more powerful when a language model replaces hand-coded mutation functions. In circle-packing optimization, an LLM generates solution variants, fitness scores rank them, and the process iterates — navigating solution spaces that gradient descent cannot traverse because no differentiable objective exists across discrete or code-based representations.
•Quality Diversity over Single-Objective Optimization: Optimizing purely for fitness score causes evolutionary systems to get trapped — a network learning a T-maze reward task performs worse than random chance by consistently choosing the smaller reward. Researchers should apply quality diversity algorithms that simultaneously reward exploration breadth and solution quality, preventing premature convergence to locally decent but globally poor strategies.
•Co-evolving Agent and Environment: Training agents against static environments produces brittle specialists. The POET algorithm and its successors evolve terrain difficulty alongside the agent, starting simple and progressively increasing complexity. This curriculum approach enables bipedal robots to eventually navigate obstacle courses they could never learn directly — a principle now extendable using LLMs to generate Unity environments via code.

Notable Moment

Risi describes a counterintuitive failure mode in plasticity research: a network trained to adapt in a T-maze consistently learned the worst possible strategy — always choosing the smaller reward — scoring below random chance, yet appearing closer to the correct solution than a network that ignored rewards entirely.

Know someone who'd find this useful?

Episode Transcript

We can take an example of how nature evolved intelligence and use evolution instead. The nice thing is it doesn't only have to be the weights like in standard back propagation, but it can be the architecture. We can start from a completely random network. The only thing it has the learning rules, but otherwise the weights are completely random. And in a few steps, the network can self organize when you use a static fixed network that is not changing the weights during its lifetime. If you cut off a leg, it will probably fail because it can't adapt. But these heavier networks, they change the weights all the time. It's basically like a continually learning, updating system where you can cut off a leg, and oftentimes it will still be able to function even though it has never seen this kind of variation during training. So we're gonna talk about evolutionary strategies, evolutionary or neuroevolution Right. The book that you guys and about your work at Secana. Sure. Yeah. Yeah. To begin, you have this new book out, Neuro Evolution. Mhmm. And it's about evolutionary strategies in, machine learning or in, AI. And it pulls together several different strategies or streams of of thought. Which are you working on most? So maybe you can start by explaining what neuroevolution is in AI. Sure. So neuroevolution is the idea of combining evolutionary algorithms with, neuroevolution, with evolution. So so the idea is instead of training networks with, you know, like gradient descent or reinforcement learning, we can take an example of how nature evolved intelligence and use evolution instead. And so you're applying genetic algorithms, evolutionary strategy, many different flavors of, evolutionary algorithms to optimize some part of the, of a neural network. And that can be The nice thing is it doesn't only have to be the weights, like in standard, backpropagation, but it can be the architecture. It can be some, learning rule. It can be hyperparameters. It can be many different parts, and it doesn't have to be differentiable. So, it's quite versatile how you can apply it. What do you mean it doesn't have to be differentiable? So, so when you typically train a neural network, with, like, supervised learning, you need to be able to differentiate through the network. So, the whole, the way you know in standard when you use an algorithm like backpropagation, what basically all machine, most machine learning is built on. Be able to differentiate through the network to know how much, based on the loss, like how well it performs, how much you change each weight in the network. And, so the network has to have certain properties like this differentiability, for it to be able to apply this algorithm. And so if you don't have that, then then it's a little bit more complicated to apply something like backpropagation to it. Evolution basically still works. So you have to have specific, like, activation functions that are differentiable. The …

Get the full transcript (10,169 words) + summary by email — free

One-time email with the complete transcript and AI summary of this episode. No account needed.

One email, no spam. We’ll also show you what SignalCast does.

Browse all Eye on AI transcripts →