86% of What Coding Agents Do Is Just Reading — Not Solving | Alexander Whedon of Subquadratic
Episode
54 min
Read time
2 min
Topics
Productivity, Remote Work, Relationships
AI-Generated Summary
Key Takeaways
- ✓Coding agent efficiency: Benchmarking frontier models on SWEBench Pro revealed that 86% of agent steps are read operations — context gathering before execution — with only 14% spent on actual problem-solving. Eliminating redundant retrieval steps through larger context windows could reduce 60-step workflows to 6-7 steps, cutting latency and token costs simultaneously.
- ✓RAG transformation, not elimination: Rather than replacing RAG entirely, large context windows allow practitioners to increase chunk sizes beyond the common 400-token limit, expand top-k retrieval from 10 to 50 results, and run 20 parallel searches simultaneously. This removes the need for $10M data transformation projects before enterprises can build AI products on raw data.
- ✓Sparse attention without quadratic overhead: DeepSeek's dynamic sparse attention still uses a full-attention selection model that exceeds the cost of sparse layers at just 52,000 tokens. SubQ's SSA performs dynamic token relationship selection within the same model doing generation, achieving 40x faster inference and 64x less compute than FlashAttention v4 at 1 million tokens.
- ✓Pretraining scale unlocks extrapolation: SubQ's 1.1 small model, pretrained extensively on 1-million-token inputs, demonstrated the ability to perform retrieval tasks at 12 million tokens during inference without any post-training exposure above 1 million tokens. This confirms that robust long-context pretraining enables capability extrapolation far beyond the training distribution ceiling.
- ✓Long-context intelligence remains asymmetric: Current frontier models perform near 100% on easy and medium code tasks at 800,000 tokens but drop well below 50% on financial document analysis at the same context length. Enterprises building on long-context models should validate performance specifically within their document type — PDFs with distributed tabular data degrade model accuracy significantly.
What It Covers
Alexander Whedon, CTO of SubQuadratic, explains how his company's sparse attention mechanism (SSA) reduces transformer compute costs by 40x at 1 million tokens, enabling multimillion-token context windows that reshape enterprise RAG architectures, coding agents, and long-horizon AI workflows without quadratic scaling penalties.
Key Questions Answered
- •Coding agent efficiency: Benchmarking frontier models on SWEBench Pro revealed that 86% of agent steps are read operations — context gathering before execution — with only 14% spent on actual problem-solving. Eliminating redundant retrieval steps through larger context windows could reduce 60-step workflows to 6-7 steps, cutting latency and token costs simultaneously.
- •RAG transformation, not elimination: Rather than replacing RAG entirely, large context windows allow practitioners to increase chunk sizes beyond the common 400-token limit, expand top-k retrieval from 10 to 50 results, and run 20 parallel searches simultaneously. This removes the need for $10M data transformation projects before enterprises can build AI products on raw data.
- •Sparse attention without quadratic overhead: DeepSeek's dynamic sparse attention still uses a full-attention selection model that exceeds the cost of sparse layers at just 52,000 tokens. SubQ's SSA performs dynamic token relationship selection within the same model doing generation, achieving 40x faster inference and 64x less compute than FlashAttention v4 at 1 million tokens.
- •Pretraining scale unlocks extrapolation: SubQ's 1.1 small model, pretrained extensively on 1-million-token inputs, demonstrated the ability to perform retrieval tasks at 12 million tokens during inference without any post-training exposure above 1 million tokens. This confirms that robust long-context pretraining enables capability extrapolation far beyond the training distribution ceiling.
- •Long-context intelligence remains asymmetric: Current frontier models perform near 100% on easy and medium code tasks at 800,000 tokens but drop well below 50% on financial document analysis at the same context length. Enterprises building on long-context models should validate performance specifically within their document type — PDFs with distributed tabular data degrade model accuracy significantly.
Notable Moment
Whedon reframes robotics as a long-context problem: teaching a robot to do laundry requires navigating a house, recognizing clothing cycles, and recalling user folding preferences — a task he estimates requires roughly 4 million tokens of context, currently impossible on edge hardware with standard attention architectures.
Episode Transcript
In transformer models, the cost of attention grows with the square of the context length. So doubling context makes a compute roughly four times more expensive. And Alex and his team have solved this issue. When we've done benchmarking the frontier models with Subench Pro, we found that 86% of the steps were actually read steps, just trying to do that context engineering before execution review, which was only the last 14%. Is this the end of RAG? If you put everything that's in your RAG knowledge base into the context window, you wouldn't need that more complicated architecture. Is that right? Yeah. It is. Although, I wouldn't say it's the end of rag just because it's too extreme. If all of these enterprises that are sitting on massive amounts of data and a lot of that data has yet to be put to use in an AI product, they're getting pushed to do, like, a $10,000,000 data transformation project before they can even start to build a product on top. And value prop that that we bring to the table is that is no longer necessary. Hi. This is Ion AI. And today, we're talking to Alex Weeden about his company SubQ, which solves a problem that plagues transformer models. In transformer models, the cost of attention grows with the square of a context length. So doubling context makes a compute roughly four times more expensive. That's what we mean by quadratic compute complexity. And Alex and his team have solved this issue. So, Alex, can you start by introducing yourself to listeners and we'll we'll go from there. Sure. I'm Alex, cofounder and CTO here at SubQuadratic. I've been in the space for about ten years. I started working with, language models since before transformers existed when LSTMs were still the hot thing. And our mission here is to always be the first to make the most computationally memory and sample efficient algorithms and build foundation models on top of those. Yeah. And can you explain you've you've tackled this quadratic complexity with sparse attention. Am I right? And can you explain what sparse attention is in simple terms and why it matters compared to standard transformer attention? Sure. So the quadratic bottleneck of attention is introduced because attention compares two token relationships. So you have 1,000 tokens, and there are 1,000 squared two token relationships or a million squared two token relationships for a million tokens. You have this quadratically go growing set of math that you're going to perform with an attention. Sparse attention says we don't actually need all of those token relationships, and so sparse attention attempts attempts to find which token relationships are important before running attention on only a subset of the token relationships. And one of the main challenges with sparse attention is that there there's always been a quality trade off. As you can imagine, like, in theory, if you're not looking at every token relationship, then you can potentially …
Get the full transcript (7,612 words) + summary by email — free
One-time email with the complete transcript and AI summary of this episode. No account needed.
One email, no spam. We’ll also show you what SignalCast does.
You just read a 3-minute summary of a 51-minute episode.
Get Eye on AI summarized like this every Monday — plus up to 2 more podcasts, free.
Pick Your Podcasts — FreeKeep Reading
More from Eye on AI
From 10 Drones a Month to Nearly 100,000 — Inside Ukraine's Largest Drone Manufacturer | Marko Kushnir, General Cherry
Sep 3 · 38 min
Software Engineering Daily
Foundation Models for Structured Data
Jun 23
More from Eye on AI
In 5 to 10 Years, Using Weapons Without AI Will Be Considered Unethical | Yaroslav Azhnyuk, The Fourth Law
Aug 31 · 53 min
The TWIML AI Podcast
Is RAG Dead? Lessons from Building AI for Tax Law with Alex Bowcut - #769
Jun 9
More from Eye on AI
We summarize every new episode. Want them in your inbox?
From 10 Drones a Month to Nearly 100,000 — Inside Ukraine's Largest Drone Manufacturer | Marko Kushnir, General Cherry
In 5 to 10 Years, Using Weapons Without AI Will Be Considered Unethical | Yaroslav Azhnyuk, The Fourth Law
Inside Ukraine's Azov Drone R&D: The Engineer Building AI Weapons 18 km From the Front Line | Alexander Palamarchuk
95% of AI Agent Projects Fail to Reach Production. Here's Why | Manoj Saxena, TrustWise
From Zero to 150 Robots in Just 20 Months | Mike LeBlanc, Foundation Future Industries
Similar Episodes
Related episodes from other podcasts
Software Engineering Daily
Jun 23
Foundation Models for Structured Data
The TWIML AI Podcast
Jun 9
Is RAG Dead? Lessons from Building AI for Tax Law with Alex Bowcut - #769
Decoder
Jan 29
Netflix is eating Hollywood — because it has to
The TWIML AI Podcast
Dec 17
Rethinking Pre-Training for Agentic AI with Aakanksha Chowdhery - #759
Huberman Lab
Nov 27
Essentials: Using Hypnosis to Enhance Mental & Physical Health & Performance | Dr. David Spiegel
Explore Related Topics
This podcast is featured in Best AI Podcasts (2026) — ranked and reviewed with AI summaries.
You're clearly into Eye on AI.
Every Monday, we deliver AI summaries of the latest episodes from Eye on AI and 192+ other podcasts. Free for one show.
Start My Monday DigestNo credit card · Unsubscribe anytime