The next big breakthrough will be AIs learning on the job

What It Covers

Dwarkesh Patel argues that AI's next capability leap requires on-the-job continual learning, explaining why current RLVR training hits hard limits and how techniques like on-policy self-distillation and "dreaming" could unlock genuine AGI-level generalization by 2027–2028.

Key Questions Answered

• •RLVR Generalization Limits: Reinforcement learning on verifiable, containerized environments works for coding and math but cannot train skills requiring real-world feedback loops — like winning court cases or building a business — because rollouts take months and cannot be parallelized or replayed from identical starting states.
• •Grindability Requirement: A domain being verifiable is insufficient for rapid AI progress; it must also be "grindable" — runnable as thousands of parallel, deterministic, replayable simulations. Computer use lags behind coding precisely because cloning real websites like Amazon at scale remains prohibitively labor-intensive today.
• •On-Policy Self-Distillation (OPSD): Rather than sparse RL rewards or full transcript replay via supervised fine-tuning, OPSD trains the base model to match per-token predictions of a context-rich "veteran" model, producing targeted weight updates that consolidate session learning without overwriting existing knowledge — superior density versus naive RL.
• •"Dreaming" as a Fourth Scaling Axis: Beyond pretraining, RL, and inference-time compute, models could spend compute generating their own RL environments simulating a specific user's real-world context, then train against them before deployment — analogous to EfficientZero's internal simulation strategy but applied to open-ended professional tasks.