Justin Solomon

AI Summary

→ WHAT IT COVERS MIT Associate Dean of Engineering Justin Solomon joins Odd Lots to examine how AI models are reshaping mathematics — from the recent AI-assisted Navier-Stokes counterexample to collapsing academic peer review systems. The conversation covers proof verification tools like Lean, credit attribution challenges, and how math education at institutions like MIT is restructuring around experiential learning rather than problem sets. → KEY INSIGHTS - **Proof verification via Lean:** AI models generate mathematical proofs but cannot reliably self-check them. The productive workflow pairs a large language model with Lean, a formal programming language that independently verifies each logical step against built-in axioms. This human-AI-software triangle produces results the math community can trust more than AI output alone, and represents the current state-of-the-art in AI-assisted theorem proving. - **Academic publishing breakdown:** Machine learning conference submissions have grown from roughly 1,000–2,000 to 60,000 annually at venues like ICLR over the past decade. This volume breaks peer review entirely — reviewers lack bandwidth to open every PDF, and anecdotal reports describe high school students being assigned as peer reviewers. Any field relying on conference publication as a quality signal should treat that signal as significantly degraded. - **Compute access as competitive advantage:** When news of progress toward the Navier-Stokes counterexample emerged, OpenAI reportedly redirected substantial compute resources to publish a proof first. Individual academic mathematicians cannot match that capacity. A $200-per-month cloud AI account already creates equity gaps among PhD students, and the gap between frontier internal models and publicly available models compounds this structural disadvantage further. - **AI operates inside the "convex hull" of existing knowledge:** Current models excel at recombining and extending known mathematical facts in novel ways but have not demonstrated the ability to generate genuinely new conjectures or theoretical frameworks from scratch. The distinction matters for AGI claims — brute-forcing a well-defined open problem differs fundamentally from identifying which problems are worth asking in the first place, a judgment that remains human-driven. - **Assessment redesign for AI-era classrooms:** Solomon's MIT course now pairs traditional problem sets with immediate follow-up quizzes using near-identical problems to verify student comprehension rather than AI capability. Projects require oral presentation components. The framing shift treats homework as weightlifting — process-oriented skill-building — rather than output production. Institutions can adapt this two-layer assessment model without abandoning mathematical rigor. - **Credit attribution in AI-assisted proofs is unresolved:** The Navier-Stokes counterexample involved years of foundational work by a Spanish mathematics team, followed by AI completing the final steps. No clear framework exists for assigning academic credit, hiring weight, or prize eligibility in such cases. The Clay Mathematics Institute's $1 million prize criteria require multi-year community verification, and OpenAI has already stated it does not need the money — removing the traditional incentive structure entirely. → NOTABLE MOMENT Solomon revealed that after testing AI tools on open problems in his own research area, the model produced a superficial result and then automatically offered to format it as a journal submission — complete with a note claiming the result appeared novel. He connects this directly to the flood of unsolicited emails he now receives from non-mathematicians seeking MIT validation of AI-generated proofs. 💼 SPONSORS [{"name": "Barclays", "url": "https://www.barclays.com"}] 🏷️ Artificial Intelligence, Mathematics Education, Proof Verification, Academic Publishing, AGI Benchmarks, Research Credit Attribution