20VC: Who Wins the Inference Market | Has Open-Source LLM Inference Peaked? | What Happens to the Cost of Compute | The Real Limits of Post-Training and RL with Erik Bernhardsson, Modal CEO

What It Covers

Erik Bernhardsson, CEO of Modal (recently valued at $15.75B on $1B ARR), breaks down the inference market structure, GPU procurement realities, open-source versus frontier model dynamics, post-training limitations, and how Modal grew 15x in twelve months by building a multi-product infrastructure platform rather than a single-commodity inference layer.

Key Questions Answered

• •Inference market structure: Selling raw tokens is a race to the bottom with zero switching costs. The defensible strategy is bundling inference with sandboxes, post-training, reinforcement learning, batch jobs, and workflow orchestration into a unified platform. Modal's Sandbox product alone reached a $300M run rate by mid-2025, roughly one-third of total revenue, after launching in 2023 with minimal early traction.
• •GPU procurement reality: GPU lead times currently run three to four months. Modal forecasts demand by extrapolating 30-40% monthly growth rates across its diversified customer base, then purchases two to three times that projected capacity. Working with 25 neo-clouds simultaneously smooths individual customer variability into predictable aggregate demand, enabling pay-per-GPU-second pricing without multi-year rectangular capacity commitments.
• •Open-source inference staying power: The claim that frontier model price drops will kill open-source inference assumes a fixed market size. Both segments are likely to grow 5-10x or more over the next twelve months. Enterprises fine-tuning open-source models gain consistency guarantees—the underlying model won't change or disappear—plus control over latency, throughput, and continuous parameter adjustment without dependency on a frontier lab's roadmap.
• •Post-training and RL limits: Reinforcement learning works well for tasks with short, verifiable feedback loops—coding, math proofs, cybersecurity—because correctness can be checked automatically inside sandboxes. Tasks with feedback loops longer than days or weeks, such as long-horizon engineering projects, remain outside current RL reach. Bernhardsson expects this boundary to shift gradually but cautions against assuming RL scales indefinitely to all knowledge work.
• •Multi-product timing discipline: Modal spent four years before its second product, Sandboxes, generated meaningful revenue. The trigger was models like Claude becoming capable enough to generate and execute full applications autonomously in mid-2025. The lesson: build infrastructure primitives that anticipate future killer apps rather than chasing current demand, then staff aggressively once product-market fit appears—Bernhardsson scaled the LLM inference team from two to thirty people within twelve months of recognizing the opportunity.
• •Talent competition and equity liquidity: New graduate engineers are receiving offers exceeding $20M from frontier labs. Modal's counter-strategy targets the 95th percentile of market compensation while emphasizing equity upside at a valuation Bernhardsson considers underpriced. Structured employee secondary programs are essential: they convert paper equity into demonstrated liquid value, giving recruiters a concrete data point that Modal compensation packages are real money, not speculative promises.