Intelligence with Everyone: RL @ MiniMax, with Olive Song, from AIE NYC & Inference by Turing Post

What It Covers

Olive Song, senior reinforcement learning researcher at MiniMax, details the training methodology behind the open-weight M2 model — a 10-billion active parameter system built for coding and agentic tasks — covering interleaved thinking, perturbation pipelines, reward hacking, and the tight developer-researcher feedback loops that shape model behavior.

Key Questions Answered

• •Interleaved Thinking Architecture: Rather than executing a single round of tool calls, MiniMax M2 alternates between thinking and tool use across tens to hundreds of turns within one user interaction. This allows the model to detect noisy or unexpected environment responses and self-correct mid-task, directly improving performance on long-horizon agentic workflows without additional human intervention.
• •Perturbation Pipeline for Generalization: Scaling tool variety alone does not produce robust agent generalization. MiniMax systematically perturbs every dimension of the model's operational space — tool definitions, system prompts, user prompts, chat templates, and tool responses — during training. This pipeline trains the model to adapt across unseen agent scaffolds rather than overfitting to familiar configurations.
• •FP32 Precision in RL Training: A debugging investigation into stagnant accuracy during reinforcement learning revealed that reduced numerical precision was creating a measurable gap between the theoretical algorithm and its implementation. Running the language model head at FP32 precision during RL training closed that gap, demonstrating that low-level engineering decisions can outweigh algorithmic choices in practice.
• •In-House Developer Feedback as Reward Signal: MiniMax embeds expert developers directly into the RL training cycle, not just evaluation. These developers define problem types — bug fixing, repo refactoring — identify trusted model behaviors, and provide precise reward signals. This creates a tighter feedback loop than external benchmarks and surfaces alignment failures, such as unsafe bash usage, before deployment.
• •Internal AI Agent for Research Monitoring: To manage the daily volume of papers, blogs, and repositories, MiniMax runs an internal agent that tracks new publications, filters by subject area, and delivers summaries to relevant researchers. Team members can then refine the agent's filtering criteria over time, effectively using agentic tooling to maintain research coverage without manual triage.