The Annual AI Slowdown Panic is Here

Today on the AI Daily Brief, the annual summer AI slowdown panic has arrived a little early this year. Before that in the headlines, a new coding benchmark that's getting rave reviews. The AI Daily Brief is a daily podcast and video covering the most important news and discussions in AI. Alright, friends. Quick announcements before we dive in. First of all, thank you to today's sponsors, KPMG, ZenCoder, Scrunch, and Bolt. To get an ad free version of the show, go to patreon.com/aidailybrief, or you can subscribe on Apple Podcasts. Reminder that it is just $3 a month for ad free. If you wanna learn more about sponsoring the show, send us a note at sponsors@aidailybrief.ai. By By the way, for anyone who is interested, we are selling many, many months ahead now. So if you think you might be, I'd encourage you to reach out. And lastly today, a quick thing, the most important way that the podcast has grown over the last few years is when people share it internally with their work colleagues. And I realize that the podcast as it is can be fairly dense and actually sort of difficult to transmit into that sort of work setting. I've got a survey up on the website right now about how I can help make that easier. It's only a couple questions. It'll take you less than a minute to do, and I would so appreciate it if you take the time to let me know how I can make AI DB for Teams work better. You can find a link right on the main page there at aidailybrief.ai. We kick off today with a new benchmark that has people pretty excited. Now if you are a regular listener, you might remember my episode from back a couple months ago called why AI needs better benchmarks. Effectively, the lament of that piece is that most of the benchmarks we have either are or getting saturated incredibly quickly. And even if they're not, are highly susceptible to gaming in a way that makes their value in terms of understanding how good a model actually is pretty low. One of the ways that this shows up is a real disconnect between what benchmarks say when a model is first released and what people go experience. One of the areas that this has been on display recently is in the realm of agentic coding, where people's lived experience with the models has been fairly different than what's suggested by the benchmarks. Well, now we have a new entrant to the field called DeepSWE. The benchmark comes from a company called DataCurve. And in their announcement, DataCurve's Serena Goh writes, on public leaderboards, top models often look relatively close in capability. DeepSWE or DeepSWE shows where they actually diverge. We wanted tasks that reflect realistic, novel engineering work. The Sweebench family scrapes existing GitHub issues and PRs, reflecting the realistic experience of developers in their day to day work. Now …