Anthropic, the Pentagon, and the Future of Autonomous Weapons

What It Covers

Paul Scharre, executive vice president at the Center for a New American Security and author of *Army of None*, explains how the US military currently uses AI in the Iran conflict, what autonomous weapons actually means, and why the Anthropic-Pentagon contract dispute centers on who sets usage rules rather than immediate deployment of killer robots.

Key Questions Answered

• •Autonomous weapons definition: The meaningful threshold is a weapon that selects its own targets without human approval, not AI-assisted planning tools. Today's military AI, including Anthropic's Claude deployed via Palantir's Maven Smart System, helps analysts query fused satellite, signals, and geolocation data — humans still approve every strike decision, though that boundary is eroding.
• •AI-human loop quality: Nominal human oversight is not meaningful oversight. When thousands of targets are processed rapidly, analysts may rubber-stamp AI outputs rather than genuinely vet them. The school strike early in the Iran conflict, attributed to outdated Defense Intelligence Agency targeting data, illustrates how data quality failures compound when AI accelerates target generation at scale.
• •Anthropic-Pentagon dispute mechanics: The core disagreement is not about autonomous weapons today but about contract language. The Pentagon's January AI strategy demanded rights to use vendor AI for any lawful purpose, conflicting with Anthropic's acceptable-use policies. OpenAI accepted similar terms, creating a dynamic where the lab with fewer safety restrictions wins the $200 million contract.
• •Why government cannot build its own AI: The Pentagon cannot replicate frontier AI internally because AI talent concentrates in commercial firms offering higher compensation, and private capital has mobilized far more investment into data centers and model training than defense budgets allow. The commercial market for AI dwarfs defense applications, making the military a relatively minor customer for leading labs.
• •Flash-crash escalation risk: Autonomous systems interacting at machine speed in contested environments risk emergent, unintended escalation analogous to algorithmic trading flash crashes. Financial markets installed circuit breakers to halt runaway algorithms; no equivalent mechanism exists in warfare. Cyberspace and drone swarms represent the most plausible near-term environments where this failure mode could trigger unintended conflict escalation.