Production-Grade AI Systems with Fred Roma

Engineering teams around the world are building AI focused applications or integrating AI features into existing products. The AI development ecosystem is maturing, which is accelerating how quickly these applications can be prototyped. However, taking AI applications to production remains a notoriously complex process. Modern AI stacks demand LLMs, embeddings, vector search, observability, new caching layers, and a constant adaptation as the landscape shifts week to week. Increasingly, the data layer has become both the foundation and the bottleneck to AI app productionization. MongoDB has been expanding beyond its core document database into a full AI ready database platform with integrated capabilities for operational data, search, real time analytics, and AI powered data retrieval. The company also recently acquired Voyage AI to provide accurate and cost effective embedding models and re rankers to its users. Fred Roma is a veteran engineer and is currently the SVP of Product and Engineering at MongoDB. He joins the show with Kevin Wall to talk about the state of AI application development, the role of vector search and re ranking, schema evolution and the LLM era, the Voyage AI acquisition, how data platforms must evolve to keep up with AI's breakneck pace, and more. Kevin Ball or Kate Ball is the vice president of engineering at Mento and an independent coach for engineers and engineering leaders. He cofounded and served as CTO for two companies, founded the San Diego JavaScript meetup, and organizes the AI in action discussion group through Latent Space. Check out the show notes to follow Keball on Twitter or LinkedIn or visit his website, keball.llc. Fred, welcome to the show. Hey, Kevin. Great to be here. Thanks for having me. Yes. I am really excited to dig in with you on this. So let's maybe start with a quick background of you and how you got to where you are today at at Mongo, and then maybe we can use that as a a way in. Yeah. Absolutely. So I I started as a software developer. It was a long time ago in France, in Paris. I evolved. I I I became a manager and worked, also on the product side. And I worked in different continents, Small startups, big companies. The biggest one was AWS. Small startup. It's probably some startup you haven't heard of. French startup. But yeah. No. No. I and and, I would say most of my career, I've been in the in the cloud, even before we call that cloud. Like more, you know, application service provider or but plugging some servers and giving access to these servers to customers. And mostly around security things like payment, identity encryption, and more lately at MongoDB on data management and AI. So I'm having some fun here. Yeah. So let's dive straight in there. We're talking about data management and AI. I think everyone's trying to figure out what is it to build an effective AI application right now with these newest tools. What's your take on …