Why PMs are suddenly valuable again | Tulsee Doshi (Head of Product, Gemini)
Episode
58 min
Read time
2 min
Topics
Career Growth, Startups, Fundraising & VC
AI-Generated Summary
Key Takeaways
- ✓Model-as-UX: The personality and behavior of an AI model constitutes its own user experience layer. PMs should develop a formal behavioral spec, translate it into rubrics, then work with researchers to embed those rubrics into reward modeling and training data — not just surface-level prompt tuning after the fact.
- ✓60-20-20 Build Framework: When building on AI models, structure your product so roughly 60% of features work reliably today, 20% are functional but imperfect with near-term model improvements visible, and 20% represent aspirational capabilities where early signal exists. This balances shippable value with positioning for model capability gains.
- ✓Eval-Building Starting Point: PMs new to writing evals should start with as few as five to twenty prompts targeting a specific capability, then manually rate outputs themselves. This small-scale approach builds intuition for what "good" looks like before scaling to automated or large-scale benchmarks.
- ✓PM Hiring Priorities for AI Roles: Hiring managers at frontier labs now prioritize three PM traits: daily hands-on model usage with formed opinions on quality, ability to build and interpret evals without engineering support, and adaptive roadmap thinking — treating two-month-old plans as drafts requiring constant re-interrogation given the pace of change.
- ✓AI Slop Management: To prevent teams from drowning in AI-generated documents, separate what AI should produce from what humans should produce. AI handles data aggregation — metrics, Reddit sentiment, dashboard summaries. Humans provide the strategic insight layer. Building structured internal skills and tools to automate data retrieval creates space for higher-order thinking.
What It Covers
Tulsee Doshi, Head of Product for Google's Gemini and frontier models, covers how AI model development differs from traditional product management, what skills PMs need now, Google's competitive strategy across the AI stack, and frameworks for building products on rapidly evolving models.
Key Questions Answered
- •Model-as-UX: The personality and behavior of an AI model constitutes its own user experience layer. PMs should develop a formal behavioral spec, translate it into rubrics, then work with researchers to embed those rubrics into reward modeling and training data — not just surface-level prompt tuning after the fact.
- •60-20-20 Build Framework: When building on AI models, structure your product so roughly 60% of features work reliably today, 20% are functional but imperfect with near-term model improvements visible, and 20% represent aspirational capabilities where early signal exists. This balances shippable value with positioning for model capability gains.
- •Eval-Building Starting Point: PMs new to writing evals should start with as few as five to twenty prompts targeting a specific capability, then manually rate outputs themselves. This small-scale approach builds intuition for what "good" looks like before scaling to automated or large-scale benchmarks.
- •PM Hiring Priorities for AI Roles: Hiring managers at frontier labs now prioritize three PM traits: daily hands-on model usage with formed opinions on quality, ability to build and interpret evals without engineering support, and adaptive roadmap thinking — treating two-month-old plans as drafts requiring constant re-interrogation given the pace of change.
- •AI Slop Management: To prevent teams from drowning in AI-generated documents, separate what AI should produce from what humans should produce. AI handles data aggregation — metrics, Reddit sentiment, dashboard summaries. Humans provide the strategic insight layer. Building structured internal skills and tools to automate data retrieval creates space for higher-order thinking.
Notable Moment
Doshi revealed that Google's skin tone evaluation system was rebuilt after discovering the widely used Fitzpatrick scale — originally designed for healthcare, not representation — skewed heavily toward lighter tones. Partnering with a Harvard professor, they replaced it with a ten-tone scale, directly improving Pixel camera and image generation outputs.
Episode Transcript
SPEAKER_01: The model isn't predictable. SPEAKER_01: It's kind of like Snape's potions class. SPEAKER_01: You can put all the ingredients in, SPEAKER_01: but you also have to expect that there will be a little bit of magic, SPEAKER_01: and that magic is going to be surprising. SPEAKER_00: As you're hiring PMs these days, is there anything you look for more and more? SPEAKER_01: You're looking for PMs who understand the vibes of our models or other models. SPEAKER_01: I want PMs who have an opinion, who are playing with these models every day. SPEAKER_01: You ideally want to stress test the model on things that you are excited about. SPEAKER_00: A lot of people feel that Google lags behind the top two labs right now. SPEAKER_01: I actually think we do want to win the frontier conversation. SPEAKER_01: I don't think we should say that Google is not participating in the frontier. SPEAKER_00: You have this framework around how far ahead to build. SPEAKER_00: You call it a 60-20-20 rule. SPEAKER_01: Traditional startup wisdom. SPEAKER_01: You're just like, build an MVP that works. SPEAKER_01: But I think you want to actually build features that aren't necessarily always working today. SPEAKER_00: It feels like every AI assistant launching these days is taking all this juicy SPEAKER_00: Google data, my Gmail, Google Calendar, my Docs. SPEAKER_00: I'm curious just why Google hasn't launched something awesome yet. SPEAKER_01: The team is working on some amazing things, SPEAKER_01: but I think you are right that there is more work we need to do here on this front. SPEAKER_00: Tulsi, thank you so much for being here. SPEAKER_00: Welcome to the podcast. SPEAKER_01: Thanks. I'm excited to be here. SPEAKER_00: You are head of product for Google's Frontier models, which includes Gemini, which people know. SPEAKER_00: But there's also a bunch of other models that people may not be familiar with. SPEAKER_00: Share the other models that are under your umbrella. SPEAKER_01: Yeah, so we have a set of models we kind of call our GenMedia models. SPEAKER_01: These are our models that focus on video editing and generation, SPEAKER_01: image editing and generation, if you've used Nano Banana, Lyria, our music model. SPEAKER_01: We also have Gemma, our suite of open models. SPEAKER_01: And then one set of models that actually falls under our Gemini umbrella, SPEAKER_01: but I feel like we don't talk about as much as our audio models. SPEAKER_01: So dialogue, audio understanding, text to speech, translation. SPEAKER_01: All a part of the portfolio of how we build FredShare models. SPEAKER_00: Okay, so I want to spend a little time on models. SPEAKER_00: Everyone sees models launch. SPEAKER_00: There's like a model launch every week, it feels like. SPEAKER_00: You have a really unique lens in being able to see these models from research SPEAKER_00: to pre-training, post-training launch. SPEAKER_00: What's something that you think people maybe misunderstand SPEAKER_00: about how models are created or something that may surprise people SPEAKER_00: …
Get the full transcript (12,641 words) + summary by email — free
One-time email with the complete transcript and AI summary of this episode. No account needed.
One email, no spam. We’ll also show you what SignalCast does.
You just read a 3-minute summary of a 55-minute episode.
Get Lenny's Podcast summarized like this every Monday — plus up to 2 more podcasts, free.
Pick Your Podcasts — FreeKeep Reading
More from Lenny's Podcast
How a 19-year-old intern built the growth engine behind this $2B AI company | Hector Serrano
Nov 1 · 87 min
Cognitive Revolution
The Model Eats the Scaffolding: DeepMind's Logan Kilpatrick & Tulsee Doshi on 3.5 Flash, Omni & More
May 20
More from Lenny's Podcast
An 86-minute guide to finding PMF | Rob Snyder (Author, The Power of Pull)
Oct 25 · 85 min
The AI Breakdown
Gemini 4 Argon, Sonnet 5.5 and What Matters with AI Models
Oct 1
Books, tools, and gear mentioned in this episode
SignalCast may earn commission on purchases via these links. As an Amazon Associate, SignalCast earns from qualifying purchases.
Gear
by Google
“Partnering with a Harvard professor, they replaced it with a ten-tone scale, directly improving Pixel camera and image generation outputs.”
More from Lenny's Podcast
We summarize every new episode. Want them in your inbox?
How a 19-year-old intern built the growth engine behind this $2B AI company | Hector Serrano
An 86-minute guide to finding PMF | Rob Snyder (Author, The Power of Pull)
Inside the black box of AI: How AI models actually think | Emmanuel Ameisen (Anthropic)
Why every company now needs to think and operate like a lab team | Josh Woodward (Google Labs, Gemini, AI Studio)
OpenAI’s Head of ChatGPT: We’re entering a new era of AI (again) | Tibo Sottiaux
Similar Episodes
Related episodes from other podcasts
Cognitive Revolution
May 20
The Model Eats the Scaffolding: DeepMind's Logan Kilpatrick & Tulsee Doshi on 3.5 Flash, Omni & More
The AI Breakdown
Oct 1
Gemini 4 Argon, Sonnet 5.5 and What Matters with AI Models
Latent Space
Feb 12
Owning the AI Pareto Frontier — Jeff Dean
The AI Breakdown
Oct 28
AI Can Now Create Worlds You Can Change in Real Time
Hard Fork
May 22
Our Field Trip to Google I/O + A Sit-Down With Sundar Pichai + System Update
Explore Related Topics
This podcast is featured in Best Product Management Podcasts (2026) — ranked and reviewed with AI summaries.
Read this week's Startups & Product Podcast Insights — cross-podcast analysis updated weekly.
You're clearly into Lenny's Podcast.
Every Monday, we deliver AI summaries of the latest episodes from Lenny's Podcast and 192+ other podcasts. Free for one show.
Start My Monday DigestNo credit card · Unsubscribe anytime