Why Medical AI Needs a Referee | Protege's Engy Ziedan

Hundreds of millions of people ask chatty peachy questions about their health. Who, if any, is making sure that the answers that are spit out is safe and correct? Models are going to be inhibited in their usefulness by the training data available for them. There is no one that's looking beyond the iceberg of, like, catastrophic failures and misalignment. What is the importance of evals in this industry? No one ever asks, like, what is the value of Uber? Like, show me the eval. But in today's AI market, there is a need for the pricing to be accurate. And without understanding really what is valuable and what's value less technology, this technology does not have a marginal cost of zero. And so it became our mission to provide safe and aligned data that would make AI useful. The right decision may also be very different a year from now versus what it looks like today. As we move into the future A medical AI model can ace thousands of test questions and still fail at a job we actually need it to do. In this episode, Daisy Wolf and Edith Steinman sit down with Protege cofounder and chief scientific officer, N. G. Zden, to unpack why healthcare AI needs a much better way to measure performance. They discuss the gap between benchmarks and real world clinical tasks, why subtle bias and misalignment may be harder to catch than catastrophic failures, and what happens when models evolve faster than the healthcare system can evaluate them. Engie also makes the case for an independent referee, one that can continuously test how AI behaves in real clinical settings, compare competing models, and identify where they actually need to improve. Welcome back to the a16z podcast. I'm Daisy Wolf, partner on a16z's bio and health team, joined by Eva Steinman, investor on our bio and health team as well. Today, we are talking with N. G. Zden, co founder and chief scientific officer of Protege and a healthcare economist, assistant professor at Indiana University whose work has been featured in the New York Times and cited by the CDC. Today, we are gonna dig into why medical AI has a measurement problem, why acing benchmarks does not make a model ready for the hospital, and how Protege is building the referee. Angie, welcome to the podcast. Thanks for having me. Angie, let's start with your background. How did you first meet the Protege team? What were you doing at the time and how has it evolved since then? Yep. I met Bobby when I was two years out of my PhD, an assistant professor at Tulane in my lonely office, and a pandemic had just hit. And I decided that I was going to write papers really fast. So I wanted data basically from two days ago, and that was impossible to obtain at the time. In his past life, he used to work for a data facilitation company, and he offered …