#330 Sebastian Risi: Why AI Should Be Grown, Not Trained
Episode
59 min
Read time
2 min
Topics
Health & Wellness, Artificial Intelligence, Software Development
AI-Generated Summary
Key Takeaways
- ✓Hebbian Plasticity over Fixed Weights: Networks trained with local Hebbian learning rules — where connection strength changes based on how often paired neurons fire together — demonstrate real-time adaptability that static networks lack. A quadrupedal robot controlled by such a network continues functioning after losing a leg, despite never encountering that scenario during training, because weights continuously update throughout operation.
- ✓Evolutionary Model Merging: Rather than training new models from scratch, evolution can identify which layers from existing pretrained models to combine. Sakana AI demonstrated this by merging a Japanese-language model with a math-specialized model, producing a single model competent in both domains — a scalable strategy for capability expansion without full retraining cycles.
- ✓LLMs as Mutation Operators: Evolutionary search becomes significantly more powerful when a language model replaces hand-coded mutation functions. In circle-packing optimization, an LLM generates solution variants, fitness scores rank them, and the process iterates — navigating solution spaces that gradient descent cannot traverse because no differentiable objective exists across discrete or code-based representations.
- ✓Quality Diversity over Single-Objective Optimization: Optimizing purely for fitness score causes evolutionary systems to get trapped — a network learning a T-maze reward task performs worse than random chance by consistently choosing the smaller reward. Researchers should apply quality diversity algorithms that simultaneously reward exploration breadth and solution quality, preventing premature convergence to locally decent but globally poor strategies.
- ✓Co-evolving Agent and Environment: Training agents against static environments produces brittle specialists. The POET algorithm and its successors evolve terrain difficulty alongside the agent, starting simple and progressively increasing complexity. This curriculum approach enables bipedal robots to eventually navigate obstacle courses they could never learn directly — a principle now extendable using LLMs to generate Unity environments via code.
What It Covers
Sebastian Risi, researcher at Sakana AI, explains neuroevolution — using evolutionary algorithms instead of gradient descent to optimize neural networks — and explores biologically inspired approaches including plastic networks, growing architectures, and combining large language models with evolutionary search to advance AI capabilities.
Key Questions Answered
- •Hebbian Plasticity over Fixed Weights: Networks trained with local Hebbian learning rules — where connection strength changes based on how often paired neurons fire together — demonstrate real-time adaptability that static networks lack. A quadrupedal robot controlled by such a network continues functioning after losing a leg, despite never encountering that scenario during training, because weights continuously update throughout operation.
- •Evolutionary Model Merging: Rather than training new models from scratch, evolution can identify which layers from existing pretrained models to combine. Sakana AI demonstrated this by merging a Japanese-language model with a math-specialized model, producing a single model competent in both domains — a scalable strategy for capability expansion without full retraining cycles.
- •LLMs as Mutation Operators: Evolutionary search becomes significantly more powerful when a language model replaces hand-coded mutation functions. In circle-packing optimization, an LLM generates solution variants, fitness scores rank them, and the process iterates — navigating solution spaces that gradient descent cannot traverse because no differentiable objective exists across discrete or code-based representations.
- •Quality Diversity over Single-Objective Optimization: Optimizing purely for fitness score causes evolutionary systems to get trapped — a network learning a T-maze reward task performs worse than random chance by consistently choosing the smaller reward. Researchers should apply quality diversity algorithms that simultaneously reward exploration breadth and solution quality, preventing premature convergence to locally decent but globally poor strategies.
- •Co-evolving Agent and Environment: Training agents against static environments produces brittle specialists. The POET algorithm and its successors evolve terrain difficulty alongside the agent, starting simple and progressively increasing complexity. This curriculum approach enables bipedal robots to eventually navigate obstacle courses they could never learn directly — a principle now extendable using LLMs to generate Unity environments via code.
Notable Moment
Risi describes a counterintuitive failure mode in plasticity research: a network trained to adapt in a T-maze consistently learned the worst possible strategy — always choosing the smaller reward — scoring below random chance, yet appearing closer to the correct solution than a network that ignored rewards entirely.
Episode Transcript
We can take an example of how nature evolved intelligence and use evolution instead. The nice thing is it doesn't only have to be the weights like in standard back propagation, but it can be the architecture. We can start from a completely random network. The only thing it has the learning rules, but otherwise the weights are completely random. And in a few steps, the network can self organize when you use a static fixed network that is not changing the weights during its lifetime. If you cut off a leg, it will probably fail because it can't adapt. But these heavier networks, they change the weights all the time. It's basically like a continually learning, updating system where you can cut off a leg, and oftentimes it will still be able to function even though it has never seen this kind of variation during training. So we're gonna talk about evolutionary strategies, evolutionary or neuroevolution Right. The book that you guys and about your work at Secana. Sure. Yeah. Yeah. To begin, you have this new book out, Neuro Evolution. Mhmm. And it's about evolutionary strategies in, machine learning or in, AI. And it pulls together several different strategies or streams of of thought. Which are you working on most? So maybe you can start by explaining what neuroevolution is in AI. Sure. So neuroevolution is the idea of combining evolutionary algorithms with, neuroevolution, with evolution. So so the idea is instead of training networks with, you know, like gradient descent or reinforcement learning, we can take an example of how nature evolved intelligence and use evolution instead. And so you're applying genetic algorithms, evolutionary strategy, many different flavors of, evolutionary algorithms to optimize some part of the, of a neural network. And that can be The nice thing is it doesn't only have to be the weights, like in standard, backpropagation, but it can be the architecture. It can be some, learning rule. It can be hyperparameters. It can be many different parts, and it doesn't have to be differentiable. So, it's quite versatile how you can apply it. What do you mean it doesn't have to be differentiable? So, so when you typically train a neural network, with, like, supervised learning, you need to be able to differentiate through the network. So, the whole, the way you know in standard when you use an algorithm like backpropagation, what basically all machine, most machine learning is built on. Be able to differentiate through the network to know how much, based on the loss, like how well it performs, how much you change each weight in the network. And, so the network has to have certain properties like this differentiability, for it to be able to apply this algorithm. And so if you don't have that, then then it's a little bit more complicated to apply something like backpropagation to it. Evolution basically still works. So you have to have specific, like, activation functions that are differentiable. The …
Get the full transcript (10,169 words) + summary by email — free
One-time email with the complete transcript and AI summary of this episode. No account needed.
One email, no spam. We’ll also show you what SignalCast does.
You just read a 3-minute summary of a 56-minute episode.
Get Eye on AI summarized like this every Monday — plus up to 2 more podcasts, free.
Pick Your Podcasts — FreeKeep Reading
More from Eye on AI
Why People Are Paying 10x More for AI - and What That Means for the Chip Market | Sid Sheth, d-Matrix
Aug 17 · 50 min
The Jordan Harbinger Show
1334: Justin Garcia | Why We Live, Cheat, Break, and Die for Love
May 28
More from Eye on AI
American Companies Have 36 Months to Go AI-Native or Get Left Behind | Drew Cukor, TWG AI
Aug 13 · 58 min
Machine Learning Street Talk
When AI Discovers The Next Transformer - Robert Lange (Sakana)
Mar 13
More from Eye on AI
We summarize every new episode. Want them in your inbox?
Why People Are Paying 10x More for AI - and What That Means for the Chip Market | Sid Sheth, d-Matrix
American Companies Have 36 Months to Go AI-Native or Get Left Behind | Drew Cukor, TWG AI
Why People Are Paying 10x More for AI | Sid Sheth, d-Matrix
AI Agents Fixing Your IT Before You Even Know Something Broke | Erhan Giral & Ryan Manning, BMC Helix
"According to NASA's Definition of Life, I'm Not Alive" - Why Nobody Can Define Life | Dr. Kate Adamala
Similar Episodes
Related episodes from other podcasts
The Jordan Harbinger Show
May 28
1334: Justin Garcia | Why We Live, Cheat, Break, and Die for Love
Machine Learning Street Talk
Mar 13
When AI Discovers The Next Transformer - Robert Lange (Sakana)
Modern Wisdom
Aug 6
The Biggest Myths About Dating (debunked) - #1133
Modern Wisdom
Jul 25
Polyvagal Theory: Why You Feel So Anxious All The Time - Dr Stephen Porges - #1128
The Diary of a CEO
Jul 23
URGENT UPDATE - Iran War Expert: 70% Chance US Sends Troops To Iran THIS WEEK! | Robert Pape
Explore Related Topics
This podcast is featured in Best AI Podcasts (2026) — ranked and reviewed with AI summaries.
Read this week's Health & Longevity Podcast Insights — cross-podcast analysis updated weekly.
You're clearly into Eye on AI.
Every Monday, we deliver AI summaries of the latest episodes from Eye on AI and 192+ other podcasts. Free for one show.
Start My Monday DigestNo credit card · Unsubscribe anytime