20VC: Taking on NVIDIA: Fractile CEO on the Future of Chips | What Everyone Gets Wrong About Capex Spending | The Winners and Losers in Neo Clouds with Walter Goodwin

What It Covers

Fractile CEO Walter Goodwin explains how his chip company raised $720M at a $7B valuation to challenge NVIDIA by stacking DRAM memory directly on top of logic dies — solving the memory bandwidth bottleneck that limits LLM inference speed — while mapping where value accrues across chips, neo clouds, frontier labs, and inference providers.

Key Questions Answered

• •Memory Wall Physics: FLOP count has scaled roughly one million times over 25 years, while memory bandwidth has only scaled 40 times — a 25,000x gap. This bottleneck, not compute power, limits LLM inference speed. A large frontier model generates at ~80 tokens per second not because of insufficient math capability, but because weights cannot be loaded from DRAM memory faster than that rate.
• •3D DRAM Stacking as Competitive Moat: Fractile's core architectural bet is stacking DRAM memory vertically on top of the logic die, replacing the limited "beachfront" of lateral copper wire connections with millions of vertical connections. This delivers SRAM-level bandwidth with DRAM-level capacity and density — eliminating the forced tradeoff that constrains every HBM-based chip including NVIDIA, TPUs, and Jalapeno.
• •Neo Cloud Commoditization: One competent neo cloud is functionally identical to another competent neo cloud. The current excitement exists only because it remains unclear which ones are competent. Investors should expect significant winners and losers to emerge, and should note that neo clouds are aggressively pursuing vertical integration — into inference serving, model routing, and custom silicon — specifically to escape this commoditization dynamic.
• •Test-Time Compute Scaling Drives Infinite Inference Demand: Solving hard problems like the Navier-Stokes equation required 80 hours of execution across 10,000 parallel agents generating billions of tokens. This "agent swarm" workload pattern — not single-shot queries — defines frontier AI deployment. Inference compute is the primary lever for increasing AI capability, meaning total spending on running models will be dramatically higher in ten years than today.
• •Broadcom Dependency Limits Chip Differentiation: Most custom AI chips — including TPU, Jalapeno, MTIA, and Matx — are rumored to use Broadcom as back-end partner, sharing the same HBM modules, process nodes, and fundamental building blocks. This means differentiation is largely cosmetic. Fractile bypasses Broadcom entirely, maintaining direct foundry relationships at TSMC, which enables non-standard architectures like 3D DRAM stacking that platform-dependent vendors cannot pursue.
• •Heterogeneous Data Centers Are a Transitional Problem: Deploying mixed chips — such as NVIDIA GPUs alongside Groq SRAM chips for prefill-decode disaggregation — creates stranded capacity risk because the optimal ratio shifts as context lengths grow and model sizes change. A fixed CapEx blend amortized over years cannot adapt. The correct long-term solution is a single chip architecture delivering high FLOPS, high bandwidth, and high memory capacity simultaneously, eliminating the need for orchestration layers.