AI Reality Check: Can LLMs “Scheme”?

What It Covers

Cal Newport deconstructs a Guardian article claiming AI chatbots are increasingly "scheming," tracing the reported 5x rise in incidents directly to the January 2026 launch of OpenClaw, an open-source DIY agent framework.

Key Questions Answered

• •Media Methodology Flaw: The UK AI Security Institute study tracking "AI scheming" pulled data exclusively from X.com tweets — not controlled experiments. A single viral February 22 tweet by Meta's Summer Yu caused the dataset's largest spike, inflating incident counts artificially.
• •LLM Mechanics vs. Planning: LLMs generate text via autoregressive token prediction — guessing one word at a time to complete a story pattern. They perform zero goal evaluation or rule-checking, meaning "bad plans" reflect statistical story-finishing, not intentional deception or misaligned scheming behavior.
• •OpenClaw as Root Cause: The 5x rise in reported AI misbehavior maps directly to OpenClaw's January 25 launch, which let non-experts build agents without commercial safeguards. Giving homemade agents unrestricted computer access predictably caused failures that generated high-engagement social media posts.
• •Coding Agents as Exception: LLM-based agents work reliably only in narrow conditions: limited action sets, well-documented training data, and external verification like compile checks and test suites. Outside coding environments, LLM-generated plans become unreliable stories mistaken for executable strategies.