AI Summary
→ WHAT IT COVERS Sebastian Risi, researcher at Sakana AI and author of *Neuroevolution*, explains why evolutionary algorithms offer a fundamentally different path to AI than gradient descent — covering plastic neural networks that rewire during operation, networks that grow from a single neuron, and how combining large language models with evolutionary search could automate scientific discovery. → KEY INSIGHTS - **Neuroevolution vs. Gradient Descent:** Rather than following a single downhill slope toward a solution, neuroevolution deploys a population of candidates across the entire search landscape simultaneously, using variation and selection to find solutions. This approach navigates non-differentiable, jagged problem spaces where backpropagation fails — making it applicable to discrete actions, novel architectures, and hyperparameter search without requiring smooth mathematical functions. - **Hebbian Plasticity for Damage Resilience:** Networks trained with local Hebbian learning rules — where connection strength updates based on how often two neurons fire together — continuously rewire during operation rather than freezing weights after training. In robotics experiments, quadruped controllers using these plastic networks maintained locomotion after leg removal, a scenario never seen during training, because the network self-organized in real time using only evolved local rules. - **Growing Networks from a Single Neuron:** Risi's Neural Developmental Program embeds a small recurrent network inside every neuron, allowing the overall network to grow from one node to thousands by having neurons communicate locally and decide when to spawn new nodes or modify connections. Tested on robotic tasks and a small MNIST variant, networks reached several thousand nodes — orders of magnitude smaller than current models, but the architecture scales without predefined structure. - **Evolutionary Model Merging for Capability Combination:** Sakana AI's evolutionary model merging uses evolutionary search to identify which layers from separate pre-trained models to combine, producing a merged model that inherits capabilities from both parents. In published experiments, a Japanese-language model and a math-specialized model were merged to produce a single model proficient in both domains — without any additional gradient-based training on combined data. - **LLMs as Mutation Operators for Scientific Search:** Replacing hand-coded genetic operators with large language models creates a powerful hybrid search system. The LLM generates candidate solutions or code variants, evolutionary selection scores them by fitness, and the best candidates seed the next generation. Applied to circle-packing optimization and early AI Scientist experiments — including a workshop-accepted paper — this loop improves automatically as the underlying language model improves. - **Co-evolving Agents and Environments via Curriculum:** The POET algorithm and its successors evolve agent and environment simultaneously, starting with simple terrain and progressively increasing difficulty. Agents that would fail on complex environments from the start succeed when scaffolded through graduated challenges. Extending this with LLM-generated Unity environments or neural network world models could allow neuroevolution to tackle significantly more complex tasks than current fixed-environment benchmarks support. → NOTABLE MOMENT Risi describes a counterintuitive failure in plasticity research: a network trained in a T-maze consistently chose the smaller reward because it learned to track and follow rewards — technically closer to the correct behavior than a network that always turned right, yet scoring worse. Traditional fitness selection would eliminate the more capable network first. 💼 SPONSORS None detected 🏷️ Neuroevolution, Evolutionary Algorithms, Neural Plasticity, Artificial Life, AI Scientific Discovery, Sakana AI