Andrej Karpathy — AGI is still a decade away

What It Covers

Andrej Karpathy explains why AGI development will take a decade, not a year, discussing current limitations in continual learning, reinforcement learning's fundamental flaws, model collapse issues, and why coding automation succeeds while other knowledge work automation struggles despite similar text-based interfaces.

Key Questions Answered

• •Reinforcement Learning Limitations: Current RL assigns credit uniformly across entire solution trajectories based on final outcomes, upweighting every token including wrong paths if the answer is correct. This creates high-variance estimators that waste computational work by sucking supervision through a straw from single reward signals across minutes of rollout.
• •Model Collapse Problem: LLMs generate outputs from collapsed distributions with low entropy, producing only three jokes when prompted repeatedly. Training on synthetic self-generated data causes further collapse because models cannot maintain the diversity and entropy needed for robust learning, similar to how humans become more rigid with age and repetitive thought patterns.
• •Cognitive Core Size: Optimal intelligence cores could operate at one billion parameters or less, compared to current trillion-parameter models. Most model size stores memorized internet facts rather than cognitive algorithms. Future systems should separate knowledge retrieval from reasoning capabilities, with models knowing what they don't know and looking up information as needed.
• •Pre-training Data Quality: Average pre-training examples are garbage, not Wall Street Journal articles but stock tickers and internet slop. Compression ratios show LLAMA 3 stores 0.07 bits per token across 15 trillion tokens, while context windows store 320 kilobytes per token, a 35 million fold difference explaining why in-context learning feels more intelligent than pre-trained knowledge.
• •Automation Progression Path: Call center work will automate before radiology because tasks are repetitive, ten-minute interactions with closed databases rather than messy multi-surface jobs. Expect autonomy sliders where AIs handle 80% of volume while delegating 20% to human supervisors managing teams of five AI agents, not instant full replacement across knowledge work.