[LIVE] Anthropic Distillation & How Models Cheat (SWE-Bench Dead) | Nathan Lambert & Sebastian Raschka
Episode
52 min
Read time
2 min
Topics
Remote Work, Investing, Fundraising & VC
AI-Generated Summary
Key Takeaways
- ✓Distillation Detection Limits: Anthropic identified distillation attempts by analyzing account patterns, request volume, and traffic shifts — MiniMax nearly halved its API traffic the moment Anthropic released a new model version. However, distinguishing distillation from legitimate large-scale evaluation or customer chatbot usage remains technically ambiguous, creating a gray zone that terms-of-service enforcement cannot cleanly resolve.
- ✓Teacher-Student Model Mismatch: The strongest model is not always the best distillation teacher. Open-weight models trained on Qwen outputs consistently outperform those trained on frontier API outputs, likely because token probability distributions must align between teacher and student. Labs should run ablations across multiple teacher models rather than defaulting to the highest-capability available model.
- ✓SWE-Bench Verified Collapse: OpenAI's audit of its own 500-task curated benchmark found 59 tasks were entirely unsolvable due to flawed test specifications — tasks that passed three rounds of human verification. Practitioners should treat any benchmark saturating above 80% across diverse model sizes as likely compromised, regardless of how many human verification rounds it underwent.
- ✓Benchmark Memorization as Canary: GPT-5's chain-of-thought reasoning on SWE-Bench tasks included knowledge of future Django API versions not available at the time the benchmark problems were written, revealing training data contamination. Benchmark designers should embed deliberately unsolvable "honeypot" tasks — problems with no valid solution — to detect memorization rather than genuine reasoning capability.
- ✓SWE-Bench Pro Structural Fixes: The replacement benchmark, SWE-Bench Pro, addresses three core flaws: it draws from more recent GitHub issues beyond the 2022–2023 window, maintains a private test set requiring answer submission rather than data download, and diversifies across more repositories and programming languages. Evaluators submitting to SWE-Bench Pro send only model outputs; Scale AI runs scoring server-side to prevent data leakage.
What It Covers
Nathan Lambert, Sebastian Raschka, and Swyx analyze two converging AI stories: Anthropic's public accusation that Chinese labs — primarily MiniMax and DeepSeek — used distributed API accounts to extract training data, and OpenAI's formal deprecation of SWE-Bench Verified after discovering 59 unsolvable tasks and model memorization of benchmark solutions.
Key Questions Answered
- •Distillation Detection Limits: Anthropic identified distillation attempts by analyzing account patterns, request volume, and traffic shifts — MiniMax nearly halved its API traffic the moment Anthropic released a new model version. However, distinguishing distillation from legitimate large-scale evaluation or customer chatbot usage remains technically ambiguous, creating a gray zone that terms-of-service enforcement cannot cleanly resolve.
- •Teacher-Student Model Mismatch: The strongest model is not always the best distillation teacher. Open-weight models trained on Qwen outputs consistently outperform those trained on frontier API outputs, likely because token probability distributions must align between teacher and student. Labs should run ablations across multiple teacher models rather than defaulting to the highest-capability available model.
- •SWE-Bench Verified Collapse: OpenAI's audit of its own 500-task curated benchmark found 59 tasks were entirely unsolvable due to flawed test specifications — tasks that passed three rounds of human verification. Practitioners should treat any benchmark saturating above 80% across diverse model sizes as likely compromised, regardless of how many human verification rounds it underwent.
- •Benchmark Memorization as Canary: GPT-5's chain-of-thought reasoning on SWE-Bench tasks included knowledge of future Django API versions not available at the time the benchmark problems were written, revealing training data contamination. Benchmark designers should embed deliberately unsolvable "honeypot" tasks — problems with no valid solution — to detect memorization rather than genuine reasoning capability.
- •SWE-Bench Pro Structural Fixes: The replacement benchmark, SWE-Bench Pro, addresses three core flaws: it draws from more recent GitHub issues beyond the 2022–2023 window, maintains a private test set requiring answer submission rather than data download, and diversifies across more repositories and programming languages. Evaluators submitting to SWE-Bench Pro send only model outputs; Scale AI runs scoring server-side to prevent data leakage.
Notable Moment
OpenAI researchers prompted competing models with only a benchmark task ID — no problem statement — and the models reproduced the full problem description and solution verbatim, confirming that benchmark content had been absorbed wholesale into model weights during pretraining from public GitHub data.
Episode Transcript
Okay. We're live. We have one person. Oh, okay. People will start trickling in. Thanks for coming to sale live number six. This is a very exciting one. I think we have a I mean, the topics are always fun with these as whatever is the topic of the day on our little rat racing minds trying to keep up with AI. But we're welcoming the latest writer that is joining the sale coalition. So I think this just means more content for sale. I think I've been a fan of Swix and a friend for a while at this point. So I'm very happy to have his content join us, and I think you've been doing great stuff recently on continuing to evolve this. So Thank you, sir. Welcome to the team. I just this is, like, my friends and and colleagues in the AI media space, and it's just great to be able to support people and keep that network closer. So Welcome to the team. Thanks for, I just wanna say, thanks for joining us. It's, really a pleasure to have you on here, Sean or Swyx. So yeah. Awesome. I just, coincidentally listened to your podcast about the Spla benchmark. So yeah. Awesome to, you know, small world, awesome to have you here. Yeah. Thanks for having me, and, yeah. Just glad to be on and chat. I've never ever done one of these Substack live things, so I'm curious how it works. Because I always think about Substack because I can use Better platform, but they wanna go multimedia. I think the live thing, before we get to technical content, is actually good because it gives it a different edge. It's just, like, a little bit sharper when you know you're live. I think we've all done a lot of podcasts, even podcasts that are unedited and put this later, but I think the live thing is a different element that can be tapped into nicely. So I don't know. Why don't we why don't we just dive into it? We're gonna start with distillation. I put I put how models cheat in the top so we can talk about benchmarks. I think Anthropic posted this pretty spicy blog post this week. I think it was essentially detailing how they found distributed distillation, quote, unquote, attacks on their services from prominent Chinese labs. And I'm very unsurprised with Anthropic calling in an attack. I think that that fits with a lot of their branding. Okay. Nice. Screen share. This is what we mean. Sean Sean Swick is such a pro. And it's like and the screen share paper was only dropped a few days ago. But, essentially, it's Anthropic is detailing how they found distributed accounts across multiple Chinese labs building state of their LOMs and described what they were doing and why Anthropic is concerned about this in their worldview of, like, AI geopolitics. And I think it's very interesting because I'm of the …
Get the full transcript (10,143 words) + summary by email — free
One-time email with the complete transcript and AI summary of this episode. No account needed.
One email, no spam. We’ll also show you what SignalCast does.
You just read a 3-minute summary of a 49-minute episode.
Get Latent Space summarized like this every Monday — plus up to 2 more podcasts, free.
Pick Your Podcasts — FreeKeep Reading
More from Latent Space
🔬“We have foundation models for language, not for physics” — Anima Anandkumar, Bren Professor of Computing
Aug 26 · 83 min
Lex Fridman Podcast
#490 – State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI
Feb 1
More from Latent Space
Simulation: the new Scaling Law — Joon Sung Park, Simile AI
Aug 21 · 69 min
Pivot
USS Lincoln Fallout, Spotlight on Natalie Harp, and Press Secretary Succession
Aug 18
Books, tools, and gear mentioned in this episode
SignalCast may earn commission on purchases via these links. As an Amazon Associate, SignalCast earns from qualifying purchases.
Tools
by Scale AI
“The replacement benchmark, SWE-Bench Pro, addresses three core flaws: it draws from more recent GitHub issues beyond the 2022–2023 window, maintains a private test set requiring answer submission rather than data download, and diversifies across more repositories and programming languages.”
“OpenAI's formal deprecation of SWE-Bench Verified after discovering 59 unsolvable tasks and model memorization of benchmark solutions.”
“The replacement benchmark, SWE-Bench Pro, addresses three core flaws: it draws from more recent GitHub issues beyond the 2022–2023 window, maintains a private test set requiring answer submission rather than data download, and diversifies across more repositories and programming languages.”
by Scale AI
“Evaluators submitting to SWE-Bench Pro send only model outputs; Scale AI runs scoring server-side to prevent data leakage.”
by OpenAI
“OpenAI's formal deprecation of SWE-Bench Verified after discovering 59 unsolvable tasks and model memorization of benchmark solutions.”
“GPT-5's chain-of-thought reasoning on SWE-Bench tasks included knowledge of future Django API versions not available at the time the benchmark problems were written, revealing training data contamination.”
More from Latent Space
We summarize every new episode. Want them in your inbox?
🔬“We have foundation models for language, not for physics” — Anima Anandkumar, Bren Professor of Computing
Simulation: the new Scaling Law — Joon Sung Park, Simile AI
🔬The BioAI Phase Shift - Matthew McPartlon & Neil Patil, Chai Discovery
The Inference Engineering Masterclass — Philip Kiely & Ali Taha, Baseten
Codex from 0 to 10M Users: Building ChatGPT Work — Akshay Nathan, OpenAI
Similar Episodes
Related episodes from other podcasts
Lex Fridman Podcast
Feb 1
#490 – State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI
Pivot
Aug 18
USS Lincoln Fallout, Spotlight on Natalie Harp, and Press Secretary Succession
The Prof G Pod
Jun 26
The Week: Iran, Brexit, SpaceX, and the Business of Sports
The Prof G Pod
Feb 18
Raging Moderates: Censoring Stephen Colbert Backfires
The Diary of a CEO
Aug 20
Konstantin Kisin’s WARNING: Our Leaders Have Lost Control, We’re Headed For Conflict And Violence!
Explore Related Topics
This podcast is featured in Best AI Podcasts (2026) — ranked and reviewed with AI summaries.
Read this week's Investing & Markets Podcast Insights — cross-podcast analysis updated weekly.
You're clearly into Latent Space.
Every Monday, we deliver AI summaries of the latest episodes from Latent Space and 192+ other podcasts. Free for one show.
Start My Monday DigestNo credit card · Unsubscribe anytime