Google VP of Product on The Future of Search and AI Mode | Robby Stein | E287
Episode
46 min
Read time
2 min
Topics
Productivity, Startups, Fundraising & VC
AI-Generated Summary
Key Takeaways
- ✓Early Product Validation: Start with 500 or fewer trusted testers before scaling, even for products targeting billions of users. The signal to watch is qualitative: when early users shift from reporting bugs to describing the product as indispensable and naturally integrated into their daily routine, that transition marks the first real validation checkpoint worth acting on.
- ✓Flat Retention as Product-Market Fit: Track day-0 cohorts through day 30, 60, and 90. Product-market fit appears when the retention curve stops declining and flattens — indicating a stable daily return probability. If the product improves over time, the curve tilts upward, signaling intensifying engagement. This J-curve pattern preceded AI Mode reaching 75 million DAU.
- ✓Depth Over Volume Metrics: For search, questions asked per session and follow-up query rate are stronger signals than passive impressions. Unlike scroll-based media apps where impressions require minimal commitment, each search query represents a deliberate user action, making query depth and return frequency more reliable indicators of genuine product value than raw traffic volume.
- ✓Iteration Conviction Over Early Results: Both Instagram Close Friends and Reels failed on initial launch — Close Friends had mistranslations and no feedback loop; Reels disappeared within Stories after one day. Each required four to five distinct rebuild cycles over multiple years. The pattern: maintain conviction based on a clear user problem, then study the small cohort where it partially works and rebuild around that behavior.
- ✓Leadership Focus Framework: Leaders should concentrate personal attention on projects where two conditions intersect — the initiative represents a five-to-ten year value opportunity, and it would not naturally progress without direct intervention due to organizational complexity or cross-team dependencies. Once a product reaches maturity, maintenance requires significantly less leadership involvement than the initial build phase.
What It Covers
Robby Stein, VP of Product at Google Search, details how Google is transforming its core search product through AI Mode, which has reached 75 million daily active users. He draws on lessons from Instagram's Stories and Reels launches to explain how to build high-conviction products inside large organizations.
Key Questions Answered
- •Early Product Validation: Start with 500 or fewer trusted testers before scaling, even for products targeting billions of users. The signal to watch is qualitative: when early users shift from reporting bugs to describing the product as indispensable and naturally integrated into their daily routine, that transition marks the first real validation checkpoint worth acting on.
- •Flat Retention as Product-Market Fit: Track day-0 cohorts through day 30, 60, and 90. Product-market fit appears when the retention curve stops declining and flattens — indicating a stable daily return probability. If the product improves over time, the curve tilts upward, signaling intensifying engagement. This J-curve pattern preceded AI Mode reaching 75 million DAU.
- •Depth Over Volume Metrics: For search, questions asked per session and follow-up query rate are stronger signals than passive impressions. Unlike scroll-based media apps where impressions require minimal commitment, each search query represents a deliberate user action, making query depth and return frequency more reliable indicators of genuine product value than raw traffic volume.
- •Iteration Conviction Over Early Results: Both Instagram Close Friends and Reels failed on initial launch — Close Friends had mistranslations and no feedback loop; Reels disappeared within Stories after one day. Each required four to five distinct rebuild cycles over multiple years. The pattern: maintain conviction based on a clear user problem, then study the small cohort where it partially works and rebuild around that behavior.
- •Leadership Focus Framework: Leaders should concentrate personal attention on projects where two conditions intersect — the initiative represents a five-to-ten year value opportunity, and it would not naturally progress without direct intervention due to organizational complexity or cross-team dependencies. Once a product reaches maturity, maintenance requires significantly less leadership involvement than the initial build phase.
Notable Moment
Stein describes the moment AI Mode first answered a genuinely difficult question correctly as resembling a perfect golf shot — a rare, fleeting proof of what the system could become. That single moment of working correctly, not sustained performance, provided enough conviction to continue iterating through months of inconsistent results.
Episode Transcript
As a founder, as a leader, everyone needs to think, what's gonna be the most important thing that if we get it right, this is potentially a five or ten year opportunity of growth, of value, of helpfulness that we can give people. Our journey with AI mode was like starting with AI overviews, building more sophisticated models, and letting you ask these more harder questions. And the first versions, nothing starts great. I would ask it, like, one or two questions that were really hard and it just nailed it. It's kinda like when you you hit a golf ball and you just hit a perfect golf shot. It all comes together. And then you don't do that again for a while, but you're kinda like, I know what is is possible to have a system that can get stuff done, book restaurant reservations for you, but it knows your tastes well. Search can be really useful for you specifically in a way that really just didn't exist many years ago. Hey. This is Carlos, CEO at Product School and your host on the product podcast. Today's guest is Robbie Stein, VP of Product at Google Search. Google Search is arguably the most significant product of the twenty first century, serving billions of users and holding over 90% of the global market share. As Alphabet surpassed 400,000,000,000 in annual revenue in 2025, search remains its engine, accounting for over 50% of that number. Robbie is currently steering the most significant shift in search history, the transition to AI. He oversees a massive portfolio including AI overviews, Google Lens and the new AI mode which has already scaled to 75,000,000 daily active users. Before returning to Google, Robbie was the head of consumer product at Instagram where he led the teams that built stories and reels. In our conversation, we skip the high level fluff and get into the weeds of how you build like a start up, even inside a giant company, and the specific metrics that actually matter. Why flat retention in your early cohorts is the only real sign of product market fit and the untold story of how colossal disasters with Instagram reels and close friends were the necessary steps to eventual global success. This is a masterclass in building products at the largest scale on Earth. Let's get into it. Welcome to the product podcast, Robbie. Thanks for having me. Okay. So you are a Google boomerang. Right? That's what they say. Yeah. Well, currently, he's the VP of product for Google Search. I'd love to learn a little bit more about kind of what happened between your previous run at Google and your current one. Yeah. Well, yeah, I started in Google in 2007 as an inspirational, program and worked on Gmail and ads. And then I left and, you know, I've always dreamed of founding a company. I wanted to try that, and, you know, I was kind of obsessed with the various …
Get the full transcript (9,755 words) + summary by email — free
One-time email with the complete transcript and AI summary of this episode. No account needed.
One email, no spam. We’ll also show you what SignalCast does.
You just read a 3-minute summary of a 43-minute episode.
Get Product School Podcast summarized like this every Monday — plus up to 2 more podcasts, free.
Pick Your Podcasts — FreeKeep Reading
More from Product School Podcast
Anthropic Member of Technical Staff on Leading Forward-Deployed Engineers, Turning Down 2x Pay, and Why Leaders Are Becoming ICs Again | Amandeep Khurana | E310
Sep 2 · 25 min
Lenny's Podcast
Inside Google's AI turnaround: The rise of AI Mode, strategy behind AI Overviews, and their vision for AI-powered search | Robby Stein (VP of Product, Google Search)
Oct 10
More from Product School Podcast
Mural CPO on How to Put AI in Multi-Player Mode for Your Team; Why AI Made Work Lonelier, Not Better | Elaina O'Mahoney
Aug 26 · 31 min
Software Engineering Daily
Reinventing the Python Notebook with Akshay Agrawal
Mar 10
More from Product School Podcast
We summarize every new episode. Want them in your inbox?
Anthropic Member of Technical Staff on Leading Forward-Deployed Engineers, Turning Down 2x Pay, and Why Leaders Are Becoming ICs Again | Amandeep Khurana | E310
Mural CPO on How to Put AI in Multi-Player Mode for Your Team; Why AI Made Work Lonelier, Not Better | Elaina O'Mahoney
Square Global Head of Product on: How to Build AI Agents People Actually Use, Why $49B Company Got Rid of General Managers | Willem Avé
How Skydio Ships Flying Robots With Just 20 Product Managers
How to Know Your AI Feature Actually Works (n8n's Founder's Metric) | Jan Oberhauser, CEO n8n
Similar Episodes
Related episodes from other podcasts
Lenny's Podcast
Oct 10
Inside Google's AI turnaround: The rise of AI Mode, strategy behind AI Overviews, and their vision for AI-powered search | Robby Stein (VP of Product, Google Search)
Software Engineering Daily
Mar 10
Reinventing the Python Notebook with Akshay Agrawal
Deep Questions with Cal Newport
Sep 7
How I’m Organizing My Life this Fall | Advice
Deep Questions with Cal Newport
Aug 10
How Do I Finish Meaningful Projects? | Monday Advice
Practical AI
Jul 17
The Future of AI Infrastructure with CoreWeave
Explore Related Topics
This podcast is featured in Best Product Management Podcasts (2026) — ranked and reviewed with AI summaries.
Read this week's Startups & Product Podcast Insights — cross-podcast analysis updated weekly.
You're clearly into Product School Podcast.
Every Monday, we deliver AI summaries of the latest episodes from Product School Podcast and 192+ other podcasts. Free for one show.
Start My Monday DigestNo credit card · Unsubscribe anytime