The next big breakthrough will be AIs learning on the job

So here's the big research bet that all the labs are making. They think that if we train AIs to accomplish millions of verifiable tasks across thousands of diverse RL environments, then we will have basically built AGI. Because this kind of training will have created a kind of problem solving agent. The kind of thing they can make progress on open ended tasks for weeks on end in the face of errors and mistakes and ambiguity. And the people who are optimistic about this vision will say that all these things that we talk about as the fundamental deficits in the current training paradigm, for example, the data inefficiency of these models or the fact that they lack continual learning, These things can just be steamrolled if you just scale training more. And the same way that all the fundamental research problems in natural language processing collapsed when you just threw enough compute into LLMs. So in the previous essay, I talked about how these models are one one millionth the sample efficient as humans. And the people who are in favor of the current training paradigm will say, look, that might be true, but this is only true during training. And training is this one time cost that is amortized across billions of sessions that a model will experience. And what really matters is how smart and general and sample efficient the model is during a session. And this has clearly been improving as we've been doing more RL training. AI agents are able to solve more and more ambitious problems over longer and longer time spans. Anybody who has used these models for coding knows that. Similarly, people would say, look, continue learning, this capability I keep harping about where the model's weights get updated based on what it's learning from deployment may simply not be necessary. Because if in context learning gets so good across longer and longer time horizons, that you don't need to distill back everything the model is learning on the job into the weights. People often say that their employees are not net productive until six months or more of them working on the job. So clearly, online learning is necessary for competence. But But what if you could just fit those six months into the context window? There's been tons of architectural innovations that dramatically increase the amount of information or the amount of context that a transformer can store. And why not think with a couple more years of progress, you might have what feels like infinitely large context windows. Okay. So before we discuss this research a bit further, I wanna step back and I wanna ask a completely tangential question, which I find actually very interesting and confusing about the nature of current AI progress. Why has progress on computer use been so much slower than other domains? Computer use is so clearly verifiable. You could ask a question like, did the desired Etsy item I ordered get …