Did AI Just “Solve” Math? (Let’s Take a Closer Look) | AI Reality Check

What It Covers

Cal Newport, a theoretical computer scientist with an Erdős number of three, analyzes OpenAI's claim that an LLM disproved Paul Erdős's 1946 planar unit distance conjecture. He separates legitimate mathematical progress from marketing hype, explaining what actually happened and what it means for AI capabilities in mathematics.

Key Questions Answered

• •AI Math Reality Check: OpenAI's LLM produced a 150-page chain-of-thought transcript, and human mathematicians manually combed through it to extract one counterexample idea, then polished it into a publishable paper. The LLM did not autonomously produce a proof — expert human labor was essential to the entire process.
• •Tributary Mental Model: AI capabilities do not rise uniformly like water covering all problems of equal difficulty. Instead, think of separate tributaries — math and coding are highly navigable, while most other domains hit dead ends quickly. Progress in discrete geometry proofs tells you nothing about AI performance in unrelated fields.
• •Why Math and Coding Are AI Sweet Spots: LLMs excel specifically in mathematics and programming because both share four traits: highly structured formal language, clear correctness verification, vast training data availability, and expert users willing to operate complex, imperfect tools. These conditions do not generalize to most professional domains.
• •Modular Architecture Beats Raw LLMs: Google DeepMind's AlphaProof-style modular system — combining tuned LLMs, formal proof verifiers like Lean, and systematic control logic — solved 9 of 353 open Erdős problems efficiently using small models. This purpose-built architecture outperforms prompting a massive general reasoning model and represents the practical future of AI-assisted mathematics.
• •AI Tools Could Double Math Productivity: Newport estimates that current AI-assisted proof exploration tools would make an applied mathematician roughly two times more effective in quality, comprehensiveness, and speed. The biggest gains come from handling tedious algebraic detail work and systematically searching proof spaces — tasks that consume disproportionate researcher time.