Brendan Foody on Teaching AI and the Future of Knowledge Work
Episode
61 min
Read time
2 min
Topics
Career Growth, Investing, Fundraising & VC
AI-Generated Summary
Key Takeaways
- ✓AI Performance Metrics: Frontier models improved 25-30% annually on economically valuable tasks, with GPT-5 scoring 64% on real-world professional work compared to GPT-4's baseline. This measures actual job automation potential rather than academic benchmarks, surveying experts on time allocation and creating corresponding evaluation rubrics.
- ✓Data Collection Strategy: The most valuable training data includes rubrics for grading model outputs and test question-answer pairs with scoring criteria, not just raw content. Models learn by attempting problems repeatedly and receiving scored feedback, similar to how professors grade essays with structured evaluation frameworks.
- ✓Labor Market Transformation: Within five years, majority of high-end knowledge workers will transition from performing tasks to training AI agents and building evaluation environments. This creates new job categories where investment bankers build testing frameworks instead of conducting analyses, applying fixed-cost knowledge investments across unlimited agent deployments.
- ✓Hiring Methodology: Effective talent assessment measures actual job skills through concrete projects rather than vibe-based conversations about background and cultural fit. Companies over-index on personal similarity instead of testing candidates' ability to perform specific deliverables like analyzing data rooms or drafting legal documents under realistic conditions.
- ✓Model Limitations Timeline: AI excels at tasks completable in chat windows but cannot yet draft emails or schedule meetings. Long-horizon tasks spanning 50-100 hours remain challenging. Foody predicts models will struggle to find mistakes when questioned by domain experts within six months, though automating the final 25% of expert capabilities will take significantly longer.
What It Covers
Brendan Foody, 22-year-old CEO of Mercor, discusses building the fastest-growing AI company by hiring experts to train frontier models, achieving 64% automation of economically valuable tasks and creating new job categories for knowledge workers.
Key Questions Answered
- •AI Performance Metrics: Frontier models improved 25-30% annually on economically valuable tasks, with GPT-5 scoring 64% on real-world professional work compared to GPT-4's baseline. This measures actual job automation potential rather than academic benchmarks, surveying experts on time allocation and creating corresponding evaluation rubrics.
- •Data Collection Strategy: The most valuable training data includes rubrics for grading model outputs and test question-answer pairs with scoring criteria, not just raw content. Models learn by attempting problems repeatedly and receiving scored feedback, similar to how professors grade essays with structured evaluation frameworks.
- •Labor Market Transformation: Within five years, majority of high-end knowledge workers will transition from performing tasks to training AI agents and building evaluation environments. This creates new job categories where investment bankers build testing frameworks instead of conducting analyses, applying fixed-cost knowledge investments across unlimited agent deployments.
- •Hiring Methodology: Effective talent assessment measures actual job skills through concrete projects rather than vibe-based conversations about background and cultural fit. Companies over-index on personal similarity instead of testing candidates' ability to perform specific deliverables like analyzing data rooms or drafting legal documents under realistic conditions.
- •Model Limitations Timeline: AI excels at tasks completable in chat windows but cannot yet draft emails or schedule meetings. Long-horizon tasks spanning 50-100 hours remain challenging. Foody predicts models will struggle to find mistakes when questioned by domain experts within six months, though automating the final 25% of expert capabilities will take significantly longer.
Notable Moment
Foody reveals his eighth-grade donut arbitrage business, buying Safeway donuts for five dollars per dozen and reselling them for two dollars each at school. After the principal shut him down, he moved operations 50 feet off campus and paid his mother 20 dollars weekly to transport inventory.
Episode Transcript
Conversations with Tyler is produced by the Mercatus Center at George Mason University bridging the gap between academic ideas and real world problems Learn more at mercatus.org. For a full transcript of every conversation enhanced with helpful links, visit conversationswithtyler.com. Hello, everyone, and welcome back to Conversations with Tyler. Today, I'm sitting here chatting with Brendan Fudi at the offices of Merkor. Merkor is an AI company, we'll get into more details soon enough, which dates from early twenty twenty three. Brendan is the CEO and cofounder. I believe he's the youngest unicorn founder ever. Mercor, by some estimates, is the fastest growing company ever. For instance, the quickest speed to $400,000,000. Brendan also at age 22 is the youngest Conversations with Tyler guest ever. My proudest achievement. There's more we'll get to soon enough. But, Brendan, welcome. Thank you so much for having me, Tyler. Excited to be here. Now I saw an ad online not too long ago from Aircor, and it said a $150 an hour for a poet. Why would you pay a poet a $150 an hour? That's a phenomenal place to start. I I think it's because so for background on what the company does, we hire all of the experts that teach the leading AI models. And so when one of the AI labs wants to teach their models how to be better at poetry, We'll find some of the best poets in the world that can help to measure success via creating evals and examples of how the model should behave. And one of the reasons that we're able to pay so well to attract the best talent is that when we have these phenomenal poets that teach the models how to do things once, they're then able to apply those skills and that knowledge across billions of users, hence, allowing us to pay a $150 an hour for some of the best poets in the world. So the poets grade the poetry of the models, or they grade the writing, or what is it they're grading? It could be some combination depending on the project. But an example might be similar to how a professor in English class would create a rubric to grade an essay or a poem that they might have for the students. We could have a poet that creates a rubric to grade, you know, how well is the model per creating whatever poetry you would like and a response that would be desirable to a given user. How do you know when you have a good poet or a great poet? It's that's so much of the challenge of it, especially with these very subjective domains in the liberal arts. Right? Is that so much of it is this question of taste, where you want some degree of consensus of, you know, different exceptional people believing that they're each doing a good job. But you probably don't want too much consensus because you also want to …
Get the full transcript (11,293 words) + summary by email — free
One-time email with the complete transcript and AI summary of this episode. No account needed.
One email, no spam. We’ll also show you what SignalCast does.
You just read a 3-minute summary of a 58-minute episode.
Get Conversations with Tyler summarized like this every Monday — plus up to 2 more podcasts, free.
Pick Your Podcasts — FreeKeep Reading
More from Conversations with Tyler
Jared Diamond on Leaders, Luck, and Irreplaceability
Sep 9 · 52 min
20VC (20 Minute VC)
20VC: Mercor: From $1M to $500M in 17 Months: The Fastest Growing Company in the World | How to Think About Margins and Revenue Sustainability in AI | Why Evaluation Benchmarks in AI are BS Today with Brendan Foody
Sep 15
More from Conversations with Tyler
Michael Moritz on Origin Stories, Exile, and Obsessive Curiosity
Sep 2 · 41 min
20VC (20 Minute VC)
20VC: Mercor CEO on Why Application Layer Companies Have No Defensibility, The Model is the Product | Token Spend Will Exceed Headcount Spend in 5 Years | The True Cost of Hiring AI Researchers in the Valley Today with Brendan Foody
Jun 1
More from Conversations with Tyler
We summarize every new episode. Want them in your inbox?
Jared Diamond on Leaders, Luck, and Irreplaceability
Michael Moritz on Origin Stories, Exile, and Obsessive Curiosity
Luke Burgis on Mimetic Desire, Catholicism, and the Age of Social Contagion
Daron Acemoglu on Liberalism, Automation, and the Educated Elite
Julia Ioffe on Russian Women, Missing Men, and the Roots of Autocracy
Similar Episodes
Related episodes from other podcasts
20VC (20 Minute VC)
Sep 15
20VC: Mercor: From $1M to $500M in 17 Months: The Fastest Growing Company in the World | How to Think About Margins and Revenue Sustainability in AI | Why Evaluation Benchmarks in AI are BS Today with Brendan Foody
20VC (20 Minute VC)
Jun 1
20VC: Mercor CEO on Why Application Layer Companies Have No Defensibility, The Model is the Product | Token Spend Will Exceed Headcount Spend in 5 Years | The True Cost of Hiring AI Researchers in the Valley Today with Brendan Foody
Techmeme Ride Home
Nov 4
Nobody Cares About AI Ads?
a16z Podcast
Aug 14
Ben Horowitz and Travis Kalanick on Building Again
Investing for Beginners
Aug 3
Birdseye View ASTS: Moonshot Potential vs. Financial Reality
Explore Related Topics
This podcast is featured in Best Finance Podcasts (2026) — ranked and reviewed with AI summaries.
Read this week's Investing & Markets Podcast Insights — cross-podcast analysis updated weekly.
You're clearly into Conversations with Tyler.
Every Monday, we deliver AI summaries of the latest episodes from Conversations with Tyler and 192+ other podcasts. Free for one show.
Start My Monday DigestNo credit card · Unsubscribe anytime