Fable is Back: Here's What You Should Try First

Today on the AI Daily Brief, Fable five is officially coming back. Before that in the headlines, the quest to cut inference costs. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. Alright, friends. Quick announcements before we dive in. First of all, thank you to today's sponsors, KPMG, robots and pencils, Blitsy, and Airtable. To To get an ad free version of the show, go to patreon.com/aidailybrief, or you can subscribe on Apple Podcasts. And if you wanna learn more about sponsoring the show, send us a note at sponsors@aidailybrief.ai. We kick off today with a story that is very of the zeitgeist that we are living in right now. OpenAI has found a way to slash their inference costs in half, sort of. This headline from the information grabbed a lot of attention, and understandably so. Everyone right now is looking for new approaches to token efficiency, and the implications of of these searches have huge impacts on the business models and the companies that are shaping AI and the larger market structures they're operating in. Now when it comes to this article specifically, the details do suggest that it might be a smaller breakthrough than it appears at first. The claim is that OpenAI researchers have discovered a new optimization technique that cut their inference requirements in half for existing models. When the technique was applied to chat g p t users who weren't signed in to the service, OpenAI was able to serve that entire user base segment on just a 100 GPUs. The OpenAI source didn't disclose what the technique was. The information speculated it could be quantization, cache optimization, batching queries, or routing queries to a lower power model. Notably, none of those techniques would improve service for OpenAI larger models without compromises. The universal truth that there is no free lunch remains, and most attempts at optimizing inference come at the expense of model quality. Now there's also the question of what it means that OpenAI is testing this technique on a tiny batch of their least engaged users. That might be a totally reasonable starting point, just the first test of many, or it could be a cautionary approach that implies that there's some risk of quality degradation. The TLDR is that while there seems to be something interesting here, we probably shouldn't treat it like some sort of silver bullet to resolve the compute crunch. Still, the information Stephanie Palazzolo is convinced that OpenAI is onto something. In an accompanying video, she said, this is a very important secret sauce for them that they don't even want to tell other OpenAI employees about, because if these things leak, it can quickly be picked up by other labs, which can also then use that to lower their costs. This is something they're holding very close to their chests. Now, many pointed to a new research paper from DeepSeek, which open sources a …