Alexander Whedon

AI Summary

→ WHAT IT COVERS Alexander Whedon, CTO of SubQuadratic, explains how his company's sparse attention mechanism (SSA) reduces transformer compute costs by 40x at 1 million tokens, enabling multimillion-token context windows that reshape enterprise RAG architectures, coding agents, and long-horizon AI workflows without quadratic scaling penalties. → KEY INSIGHTS - **Coding agent efficiency:** Benchmarking frontier models on SWEBench Pro revealed that 86% of agent steps are read operations — context gathering before execution — with only 14% spent on actual problem-solving. Eliminating redundant retrieval steps through larger context windows could reduce 60-step workflows to 6-7 steps, cutting latency and token costs simultaneously. - **RAG transformation, not elimination:** Rather than replacing RAG entirely, large context windows allow practitioners to increase chunk sizes beyond the common 400-token limit, expand top-k retrieval from 10 to 50 results, and run 20 parallel searches simultaneously. This removes the need for $10M data transformation projects before enterprises can build AI products on raw data. - **Sparse attention without quadratic overhead:** DeepSeek's dynamic sparse attention still uses a full-attention selection model that exceeds the cost of sparse layers at just 52,000 tokens. SubQ's SSA performs dynamic token relationship selection within the same model doing generation, achieving 40x faster inference and 64x less compute than FlashAttention v4 at 1 million tokens. - **Pretraining scale unlocks extrapolation:** SubQ's 1.1 small model, pretrained extensively on 1-million-token inputs, demonstrated the ability to perform retrieval tasks at 12 million tokens during inference without any post-training exposure above 1 million tokens. This confirms that robust long-context pretraining enables capability extrapolation far beyond the training distribution ceiling. - **Long-context intelligence remains asymmetric:** Current frontier models perform near 100% on easy and medium code tasks at 800,000 tokens but drop well below 50% on financial document analysis at the same context length. Enterprises building on long-context models should validate performance specifically within their document type — PDFs with distributed tabular data degrade model accuracy significantly. → NOTABLE MOMENT Whedon reframes robotics as a long-context problem: teaching a robot to do laundry requires navigating a house, recognizing clothing cycles, and recalling user folding preferences — a task he estimates requires roughly 4 million tokens of context, currently impossible on edge hardware with standard attention architectures. 💼 SPONSORS None detected 🏷️ Sparse Attention, Long-Context LLMs, Enterprise RAG, Coding Agents, AI Infrastructure