#330 Sebastian Risi: Why AI Should Be Grown, Not Trained

What It Covers

Sebastian Risi, researcher at Sakana AI, explains neuroevolution — using evolutionary algorithms instead of gradient descent to optimize neural networks — and explores biologically inspired approaches including plastic networks, growing architectures, and combining large language models with evolutionary search to advance AI capabilities.

Key Questions Answered

• •Hebbian Plasticity over Fixed Weights: Networks trained with local Hebbian learning rules — where connection strength changes based on how often paired neurons fire together — demonstrate real-time adaptability that static networks lack. A quadrupedal robot controlled by such a network continues functioning after losing a leg, despite never encountering that scenario during training, because weights continuously update throughout operation.
• •Evolutionary Model Merging: Rather than training new models from scratch, evolution can identify which layers from existing pretrained models to combine. Sakana AI demonstrated this by merging a Japanese-language model with a math-specialized model, producing a single model competent in both domains — a scalable strategy for capability expansion without full retraining cycles.
• •LLMs as Mutation Operators: Evolutionary search becomes significantly more powerful when a language model replaces hand-coded mutation functions. In circle-packing optimization, an LLM generates solution variants, fitness scores rank them, and the process iterates — navigating solution spaces that gradient descent cannot traverse because no differentiable objective exists across discrete or code-based representations.
• •Quality Diversity over Single-Objective Optimization: Optimizing purely for fitness score causes evolutionary systems to get trapped — a network learning a T-maze reward task performs worse than random chance by consistently choosing the smaller reward. Researchers should apply quality diversity algorithms that simultaneously reward exploration breadth and solution quality, preventing premature convergence to locally decent but globally poor strategies.
• •Co-evolving Agent and Environment: Training agents against static environments produces brittle specialists. The POET algorithm and its successors evolve terrain difficulty alongside the agent, starting simple and progressively increasing complexity. This curriculum approach enables bipedal robots to eventually navigate obstacle courses they could never learn directly — a principle now extendable using LLMs to generate Unity environments via code.