Robby Stein

AI Summary

→ WHAT IT COVERS Robby Stein, VP of Product at Google Search, details how Google is transforming its core search product through AI Mode, which has reached 75 million daily active users. He draws on lessons from Instagram's Stories and Reels launches to explain how to build high-conviction products inside large organizations. → KEY INSIGHTS - **Early Product Validation:** Start with 500 or fewer trusted testers before scaling, even for products targeting billions of users. The signal to watch is qualitative: when early users shift from reporting bugs to describing the product as indispensable and naturally integrated into their daily routine, that transition marks the first real validation checkpoint worth acting on. - **Flat Retention as Product-Market Fit:** Track day-0 cohorts through day 30, 60, and 90. Product-market fit appears when the retention curve stops declining and flattens — indicating a stable daily return probability. If the product improves over time, the curve tilts upward, signaling intensifying engagement. This J-curve pattern preceded AI Mode reaching 75 million DAU. - **Depth Over Volume Metrics:** For search, questions asked per session and follow-up query rate are stronger signals than passive impressions. Unlike scroll-based media apps where impressions require minimal commitment, each search query represents a deliberate user action, making query depth and return frequency more reliable indicators of genuine product value than raw traffic volume. - **Iteration Conviction Over Early Results:** Both Instagram Close Friends and Reels failed on initial launch — Close Friends had mistranslations and no feedback loop; Reels disappeared within Stories after one day. Each required four to five distinct rebuild cycles over multiple years. The pattern: maintain conviction based on a clear user problem, then study the small cohort where it partially works and rebuild around that behavior. - **Leadership Focus Framework:** Leaders should concentrate personal attention on projects where two conditions intersect — the initiative represents a five-to-ten year value opportunity, and it would not naturally progress without direct intervention due to organizational complexity or cross-team dependencies. Once a product reaches maturity, maintenance requires significantly less leadership involvement than the initial build phase. → NOTABLE MOMENT Stein describes the moment AI Mode first answered a genuinely difficult question correctly as resembling a perfect golf shot — a rare, fleeting proof of what the system could become. That single moment of working correctly, not sustained performance, provided enough conviction to continue iterating through months of inconsistent results. 💼 SPONSORS None detected 🏷️ AI Search, Product-Market Fit, Google AI Mode, Product Retention Metrics, Large-Scale Product Development