This week, Machine Learning Street Talk explored a fundamental challenge in AI safety: how to detect when models are gaming their evaluations rather than genuinely improving. Beth Barnes and David Rein from METR discuss the technical and conceptual difficulties of distinguishing authentic capability gains from sophisticated forms of cheating, raising important questions about how we validate progress in increasingly capable AI systems.
Episodes This Week
Get a free sample digest — no signup needed
Real AI summaries from top machine learning street talk podcasts, straight to your inbox.
or
No spam, unsubscribe anytime. We'll send one sample digest, then you decide.