20VC: "Anti-Data Centres is a Chinese Psyop" | How Many Planned Data Centers Will Actually Get Built? | Is Energy AI's Biggest Bottleneck? With Thomas Sohmers, Co-Founder @ Positron

What It Covers

Thomas Sohmers, co-founder of Positron AI (recently valued at $5B after an $875M Series C), breaks down the technical and geopolitical realities shaping AI infrastructure — covering inference economics, KV caching mechanics, data center politics, frontier model scaling, and why anti-data-center sentiment may serve Chinese strategic interests.

Key Questions Answered

• •Inference vs. Training Economics: Inference is memory-bound, not compute-bound — unlike training, each token requires reading all model weights sequentially, making parallelization impossible. This distinction matters for hardware investment: training scales with FLOP increases, but inference scales with memory bandwidth. Between 2014–2024, GPU FLOPs improved 120x while memory bandwidth improved only 17x, creating a structural bottleneck that defines the entire inference hardware market.
• •Token Margin Reality: Frontier AI companies are far more profitable than public perception suggests. Anthropic reportedly operates at 80% gross margin on API revenue. Cached tokens cost roughly one-thousandth of uncached tokens to process, yet providers charge near-standard rates for cache reads — generating outsized margin. If OpenAI and Anthropic stopped training today, both would become massively profitable overnight, undermining the "burning cash" narrative.
• •KV Cache Scale Problem: At long context lengths, individual user session data can exceed total model weight storage. A 10-trillion-parameter model carries roughly 5TB of weights, but a single user session at 100GB means just 50 concurrent users generate more context data than the model itself. Operators must tier storage across accelerator memory, host RAM, and NVMe flash — each tier adding latency and architectural complexity that defines inference system design.
• •Data Center Anti-Sentiment as Strategic Vulnerability: Bipartisan opposition to data centers in the US is built on factually incorrect premises — a single In-N-Out burger location uses more water than major US data centers, and golf courses consume orders of magnitude more. Meanwhile, China faces no equivalent political resistance and is adding gigawatts of new capacity. Republican governors who previously supported data center growth are reversing positions based on constituent misinformation, creating a concrete competitive disadvantage.
• •Pacing the Frontier Carries Concentration Risk: Regulatory frameworks limiting AI compute — such as restrictions on matrix multiplications or mandatory audits — risk concentrating capability among a small number of state-aligned actors. Sohmers argues this mirrors historical patterns where those advocating for regulation assume they will become the regulators. The more likely outcome is bureaucratic capture that halts progress while leaving existing capability concentrated in governments and a handful of corporations.
• •Local Models Increase, Not Decrease, Cloud Token Volume: The assumption that on-device or enterprise-hosted smaller models reduce demand for frontier API calls is likely inverted. A local model continuously processing email, calendar, and messages will autonomously generate far more upstream requests to cloud-hosted frontier models than a human manually prompting ChatGPT. The bottleneck today is human initiation of requests — removing that bottleneck through local agents multiplies total token consumption across the entire stack.