An audio version of my blog post, Thoughts on AI progress (Dec 2025)

I'm confused why some people have super short timelines, yet at the same time, are bullish on scaling up reinforcement learning atop LLMs. If we're actually close to a human like learner, then this whole approach of training on verifiable outcomes is doomed. Now, currently, the labs are trying to bake in a bunch of skills into these models through mid training. There's an entire supply chain of companies that are building RL environments, which teach the model how to navigate a web browser or use Excel to build financial models. Now either these models will soon learn on the job in a self directed way, which will make all this free making pointless, or they won't, which means that AGI is not imminent. Humans don't have to go through the special training phase or they need to rehearse every single piece of software that they might ever need to use on the job. Baron Millage made an interesting point about this in a recent blog post he wrote. He writes, quote, when we see frontier models improving at various benchmarks, we should think not just about the increased scale and the clever ML research ideas, but the billions of dollars that are paid to PhDs, MDs and other experts to write questions and provide example answers and reasoning targeting these precise capabilities. You can see this tension most vividly in robotics. In some fundamental science, robotics is an algorithms problem, not a hardware or a data problem. With very little training, a human can learn how to teleoperate current hardware to do useful work. So if you actually had a human like learner, robotics would be, in large part, a solved problem. But the fact that we don't have such a learner makes it necessary to go out into a thousand different homes and practice a million times on how to pick up dishes or fold laundry. Now, one current argument I've heard from the people who think we're gonna have a takeoff within the next five years is that we have to do all this kludgy RL in service of building a superhuman AI researcher. And then the million copies of this automated iliac can go figure out how to solve robust and efficient learning from experience. This just gives me the vibes of that old joke, we're losing money on every sale, but we'll make it up in volume. Somehow, this automated researcher is gonna figure out the algorithm for AGI, which is a problem that humans have been banging their head against for the better half of a century, while not having the basic learning capabilities that children have. I find it super implausible. Besides, even if that's what you believe, it doesn't describe how the labs are approaching reinforcement learning from verifiable reward. You don't need to pre bake in a consultant skill at crafting PowerPoint slides in order to automate Ilia. So clearly, the lab's actions hint at a worldview where these models will …