Skip to main content
Conversations with Tyler

Brendan Foody on Teaching AI and the Future of Knowledge Work

61 min episode · 2 min read
·

Episode

61 min

Read time

2 min

Topics

Career Growth, Investing, Fundraising & VC

AI-Generated Summary

Key Takeaways

  • AI Performance Metrics: Frontier models improved 25-30% annually on economically valuable tasks, with GPT-5 scoring 64% on real-world professional work compared to GPT-4's baseline. This measures actual job automation potential rather than academic benchmarks, surveying experts on time allocation and creating corresponding evaluation rubrics.
  • Data Collection Strategy: The most valuable training data includes rubrics for grading model outputs and test question-answer pairs with scoring criteria, not just raw content. Models learn by attempting problems repeatedly and receiving scored feedback, similar to how professors grade essays with structured evaluation frameworks.
  • Labor Market Transformation: Within five years, majority of high-end knowledge workers will transition from performing tasks to training AI agents and building evaluation environments. This creates new job categories where investment bankers build testing frameworks instead of conducting analyses, applying fixed-cost knowledge investments across unlimited agent deployments.
  • Hiring Methodology: Effective talent assessment measures actual job skills through concrete projects rather than vibe-based conversations about background and cultural fit. Companies over-index on personal similarity instead of testing candidates' ability to perform specific deliverables like analyzing data rooms or drafting legal documents under realistic conditions.
  • Model Limitations Timeline: AI excels at tasks completable in chat windows but cannot yet draft emails or schedule meetings. Long-horizon tasks spanning 50-100 hours remain challenging. Foody predicts models will struggle to find mistakes when questioned by domain experts within six months, though automating the final 25% of expert capabilities will take significantly longer.

What It Covers

Brendan Foody, 22-year-old CEO of Mercor, discusses building the fastest-growing AI company by hiring experts to train frontier models, achieving 64% automation of economically valuable tasks and creating new job categories for knowledge workers.

Key Questions Answered

  • AI Performance Metrics: Frontier models improved 25-30% annually on economically valuable tasks, with GPT-5 scoring 64% on real-world professional work compared to GPT-4's baseline. This measures actual job automation potential rather than academic benchmarks, surveying experts on time allocation and creating corresponding evaluation rubrics.
  • Data Collection Strategy: The most valuable training data includes rubrics for grading model outputs and test question-answer pairs with scoring criteria, not just raw content. Models learn by attempting problems repeatedly and receiving scored feedback, similar to how professors grade essays with structured evaluation frameworks.
  • Labor Market Transformation: Within five years, majority of high-end knowledge workers will transition from performing tasks to training AI agents and building evaluation environments. This creates new job categories where investment bankers build testing frameworks instead of conducting analyses, applying fixed-cost knowledge investments across unlimited agent deployments.
  • Hiring Methodology: Effective talent assessment measures actual job skills through concrete projects rather than vibe-based conversations about background and cultural fit. Companies over-index on personal similarity instead of testing candidates' ability to perform specific deliverables like analyzing data rooms or drafting legal documents under realistic conditions.
  • Model Limitations Timeline: AI excels at tasks completable in chat windows but cannot yet draft emails or schedule meetings. Long-horizon tasks spanning 50-100 hours remain challenging. Foody predicts models will struggle to find mistakes when questioned by domain experts within six months, though automating the final 25% of expert capabilities will take significantly longer.

Notable Moment

Foody reveals his eighth-grade donut arbitrage business, buying Safeway donuts for five dollars per dozen and reselling them for two dollars each at school. After the principal shut him down, he moved operations 50 feet off campus and paid his mother 20 dollars weekly to transport inventory.

Know someone who'd find this useful?

Episode Transcript

Conversations with Tyler is produced by the Mercatus Center at George Mason University bridging the gap between academic ideas and real world problems Learn more at mercatus.org. For a full transcript of every conversation enhanced with helpful links, visit conversationswithtyler.com. Hello, everyone, and welcome back to Conversations with Tyler. Today, I'm sitting here chatting with Brendan Fudi at the offices of Merkor. Merkor is an AI company, we'll get into more details soon enough, which dates from early twenty twenty three. Brendan is the CEO and cofounder. I believe he's the youngest unicorn founder ever. Mercor, by some estimates, is the fastest growing company ever. For instance, the quickest speed to $400,000,000. Brendan also at age 22 is the youngest Conversations with Tyler guest ever. My proudest achievement. There's more we'll get to soon enough. But, Brendan, welcome. Thank you so much for having me, Tyler. Excited to be here. Now I saw an ad online not too long ago from Aircor, and it said a $150 an hour for a poet. Why would you pay a poet a $150 an hour? That's a phenomenal place to start. I I think it's because so for background on what the company does, we hire all of the experts that teach the leading AI models. And so when one of the AI labs wants to teach their models how to be better at poetry, We'll find some of the best poets in the world that can help to measure success via creating evals and examples of how the model should behave. And one of the reasons that we're able to pay so well to attract the best talent is that when we have these phenomenal poets that teach the models how to do things once, they're then able to apply those skills and that knowledge across billions of users, hence, allowing us to pay a $150 an hour for some of the best poets in the world. So the poets grade the poetry of the models, or they grade the writing, or what is it they're grading? It could be some combination depending on the project. But an example might be similar to how a professor in English class would create a rubric to grade an essay or a poem that they might have for the students. We could have a poet that creates a rubric to grade, you know, how well is the model per creating whatever poetry you would like and a response that would be desirable to a given user. How do you know when you have a good poet or a great poet? It's that's so much of the challenge of it, especially with these very subjective domains in the liberal arts. Right? Is that so much of it is this question of taste, where you want some degree of consensus of, you know, different exceptional people believing that they're each doing a good job. But you probably don't want too much consensus because you also want to …

Get the full transcript (11,293 words) + summary by email — free

One-time email with the complete transcript and AI summary of this episode. No account needed.

One email, no spam. We’ll also show you what SignalCast does.

Browse all Conversations with Tyler transcripts →

You just read a 3-minute summary of a 58-minute episode.

Get Conversations with Tyler summarized like this every Monday — plus up to 2 more podcasts, free.

Pick Your Podcasts — Free

Keep Reading

More from Conversations with Tyler

We summarize every new episode. Want them in your inbox?

Similar Episodes

Related episodes from other podcasts

Explore Related Topics

This podcast is featured in Best Finance Podcasts (2026) — ranked and reviewed with AI summaries.

Read this week's Investing & Markets Podcast Insights — cross-podcast analysis updated weekly.

You're clearly into Conversations with Tyler.

Every Monday, we deliver AI summaries of the latest episodes from Conversations with Tyler and 192+ other podcasts. Free for one show.

Start My Monday Digest

No credit card · Unsubscribe anytime