It's Crunch Time: Ajeya Cotra on RSI & AI-Powered AI Safety Work, from the 80,000 Hours Podcast

What It Covers

Ajeya Cotra, AI risk researcher at METR and former Open Philanthropy technical safety grant-maker, outlines her framework for "crunch time" — the window when AI systems become capable enough to dramatically accelerate their own R&D but remain partially controllable. She argues this period may already be beginning and requires urgent transparency measures, capability monitoring, and redirecting AI labor toward safety research.

Key Questions Answered

• •AGI Definition Drift: The mainstream conflation of "AGI" with incremental improvement creates dangerous complacency. At a 2024 DealBook panel, seven of ten participants predicted AGI by 2030 yet eight of ten expected AI to create more jobs than it destroys over the same decade. This contradiction reveals that most people implicitly expect AGI to produce only 2% annual growth-style change, not the civilizational transformation Cotra considers plausible by 2050.
• •Intelligence Explosion Monitoring: To detect a runaway intelligence explosion early, track AI adoption across the full chip manufacturing stack — not just software benchmarks. Cotra recommends measuring the fraction of pull requests written and reviewed by AI with minimal human involvement, internal RCT results on productivity, and self-reported speed-ups from power users. These real-world productivity signals matter more than benchmark scores, which saturate predictably in S-curves and consistently overestimate real-world performance.
• •Transparency Requirements: Frontier AI labs should publish their highest internal benchmark scores on a fixed quarterly calendar cadence, independent of product releases. This matters because dangerous capabilities can emerge from purely internal deployment — a lab could use a powerful internal agent to accelerate R&D without ever releasing it publicly. Current practice of publishing model cards only at product launch creates a blind spot if internal models substantially outpace public releases.
• •Crunch Time Strategy: When an intelligence explosion begins, the optimal response is redirecting AI labor from further capability acceleration toward protective work: patching cyber vulnerabilities before bad actors exploit them, scaling biodefense infrastructure including pathogen detection and PPE manufacturing, and using AI to improve collective decision-making and policy coordination. Cotra frames this not as a binary pause but as a spectrum — shifting from 100% of AI labor toward capabilities to a meaningful fraction toward defense.
• •Recursive Safety Plan: All three major frontier labs — OpenAI, Anthropic, and Google DeepMind — have publicly stated safety plans that rely on using each AI generation to help align and control the next. The critical failure mode Cotra identifies is not that alignment proves technically impossible, but that competitive pressure causes labs to allocate only a token fraction of AI inference compute toward safety work rather than the substantial redirection the plan requires to function.
• •Capability Ordering Risk: The crunch time strategy fails if AI capabilities arrive in an unlucky sequence — specifically, if systems become highly specialized at AI R&D before developing broad enough capabilities to assist with safety research, biodefense, or governance. A narrow savant model that only improves training efficiency could accelerate toward a general superintelligence without providing any usable labor for protective work during the transition window, leaving humans with no useful tool despite having early warning.
• •Forecasting as Early Warning: The Forecasting Research Institute's LEAP panel (Longitudinal Experts on AI) surveys roughly 100-200 AI experts, economists, and superforecasters on granular six-month, one-year, and five-year predictions — including real-world indicators like whether companies report slowing hiring due to AI. Cotra considers this more flexible and actionable than benchmarks alone, because it connects near-term observable signals to longer-run worldviews and creates a public record for checking who was right over time.