[State of Evals] LMArena's $1.7B Vision — Anastasios Angelopoulos, LMArena

What It Covers

Anastasios Angelopoulos from Arena discusses their $100M funding round, platform economics serving tens of millions of monthly conversations, response to the Cohere leaderboard illusion controversy, principles for maintaining evaluation integrity, and expansion into specialized arenas for code, video, and occupational categories while managing one of AI's largest consumer communities.

Key Questions Answered

• •Platform Scale Economics: Arena processes mid-tens of millions of conversations monthly across 250 million total conversations, funding all inference at standard enterprise rates. The platform maintains 25 percent software developer usage even at scale, with approximately half of users now logged in, enabling demographic analysis through surveys and prompt distribution patterns to understand real user composition.
• •Leaderboard Integrity Principles: Arena treats its public leaderboard as a loss leader charity that cannot be paid for placement or removal. Model providers cannot pay to appear, improve rankings, or remove poor-performing models. Every released model receives statistically sound scores from millions of global votes, maintaining transparent evaluation independent of commercial relationships or provider preferences.
• •Prerelease Testing Strategy: Arena conducts prerelease model testing with secret codenames that drives massive user engagement and market impact. The Nano Banana launch changed Google's market share and moved billions in stock value. This community-loved approach provides early model feedback while generating viral moments, though critics incorrectly claimed it was undisclosed despite long-standing transparency.
• •Vertical Specialization Expansion: Arena now exposes occupational and expert categories across medicine, legal, business, finance, accounting, creative, and marketing verticals. Single-digit percentages of their millions-strong user base in each vertical provides sufficient scale to show model performance differences across professional use cases, moving beyond general-purpose evaluation to domain-specific benchmarks.
• •Consumer Retention Mechanics: Persistent conversation history drives significant user retention in consumer AI products. Arena learned that users are earned daily and remain fickle, requiring constant value delivery. Sign-in functionality with history persistence represents a simple but effective retention mechanism, though building dominant consumer products at ChatGPT scale requires extraordinary execution and luck beyond current reach.