Skip to main content
a16z Podcast

The Missing Layer for Personal AI | First Person Labs’ Ken Liu

33 min episode · 2 min read
·
Ken Liu

Episode

33 min

Read time

2 min

Topics

Productivity, Relationships, Startups

AI-Generated Summary

Key Takeaways

  • ✓Portable Context Layer: Rather than re-explaining preferences to every AI agent, users can point tools like Grok or Codex to their Second via MCP API. The agent queries Second directly, retrieving preferences such as airline seat choices, arrival timing, and travel class without any manual prompting from the user each session.
  • ✓Proactive vs. Reactive Productivity: Second distinguishes between reactive alerts (one email triggers one notification) and reflective productivity, where the system connects events across months to surface relevant context unprompted. Users report receiving unsolicited alerts about changing conditions—such as weather updates for planned events—without ever configuring a specific trigger or request.
  • ✓Confidential Inference Architecture: Second addresses data privacy through two mechanisms: local-only storage where data never leaves the device, and confidential computing via NVIDIA GPU hardware attestation. Unlike zero-data-retention policies, which rely on provider promises, hardware-based confidential inference is technically provable—no operator can access data running inside the attested container.
  • ✓Token Cost Trajectory: Running continuous 24/7 background agents currently costs roughly $1,000 per day using Anthropic Opus, or approximately $60 per day using DeepSeek Flash. Token prices have already dropped over 50% since First Person Labs launched, making the product viable for busy professionals now and broadly accessible within a foreseeable timeframe.
  • ✓Mixture-of-Models Architecture: Second routes tasks to different models based on complexity and frequency. Simple semantic classification tasks use lightweight models; high-reasoning queries like identifying a user's weekly goals use frontier models but run infrequently. Retrieval tasks route to Gemini, which Liu identifies as particularly well-suited for dynamic, real-time personal data retrieval.

What It Covers

First Person Labs co-founder Ken Liu presents Second, a personal AI layer that builds a continuous model of a user's preferences, priorities, relationships, and digital activity. Second functions as portable context that any AI agent can query, removing repetitive onboarding and enabling proactive, unprompted assistance across tools and devices.

Key Questions Answered

  • •Portable Context Layer: Rather than re-explaining preferences to every AI agent, users can point tools like Grok or Codex to their Second via MCP API. The agent queries Second directly, retrieving preferences such as airline seat choices, arrival timing, and travel class without any manual prompting from the user each session.
  • •Proactive vs. Reactive Productivity: Second distinguishes between reactive alerts (one email triggers one notification) and reflective productivity, where the system connects events across months to surface relevant context unprompted. Users report receiving unsolicited alerts about changing conditions—such as weather updates for planned events—without ever configuring a specific trigger or request.
  • •Confidential Inference Architecture: Second addresses data privacy through two mechanisms: local-only storage where data never leaves the device, and confidential computing via NVIDIA GPU hardware attestation. Unlike zero-data-retention policies, which rely on provider promises, hardware-based confidential inference is technically provable—no operator can access data running inside the attested container.
  • •Token Cost Trajectory: Running continuous 24/7 background agents currently costs roughly $1,000 per day using Anthropic Opus, or approximately $60 per day using DeepSeek Flash. Token prices have already dropped over 50% since First Person Labs launched, making the product viable for busy professionals now and broadly accessible within a foreseeable timeframe.
  • •Mixture-of-Models Architecture: Second routes tasks to different models based on complexity and frequency. Simple semantic classification tasks use lightweight models; high-reasoning queries like identifying a user's weekly goals use frontier models but run infrequently. Retrieval tasks route to Gemini, which Liu identifies as particularly well-suited for dynamic, real-time personal data retrieval.

Notable Moment

Liu describes a cost calculation that reframes the product's current market: running ten parallel background agents at frontier model pricing reaches $1,000 per day per user, which immediately clarifies why Second targets high-output professionals first, with broader consumer access contingent on continued token price compression.

Know someone who'd find this useful?

Episode Transcript

People are underestimating how much the computer already knows. You no longer have to explain anything to your coding agents. You can just screenshot anything and just say fix, and then you would know what you're fixing. The question then is like, well, can you just like analyze the heck out of it? What context does Grokbot not have that first person that would allow? One of the biggest bognaments right now. Like, people have to ask. You're not supposed to ask these things. They're super intelligent. But if you have a user model that just builds up by itself and then these things just keep analyzing in the background, then you can share that over instantly. We focus on essentially creating a model of a person, and then put that person in control. When people think about this thing sees everything you do, there's an immediate sort of like, woah. If you don't trust, let's say, us, in principle, you should be able to bring your own tokens to then analyze data that's on your laptop locally. Can you talk to what you think are like the most surprising use cases that you found? For development, it turns out that Every AI agent you use is trying to learn who you are. What if that context belonged to you instead? In this episode, Martin Casado and Joel De La Garza sit down with First Person Labs co founder Ken Liu to introduce Second. Second builds a model of your preferences, priorities, relationships, and digital activity that can move across the AI tools you use. They get into how it works, the security challenges of giving AI access to such personal context, and why Ken thinks the future of personal AI may be less about asking agents what to do and more about understanding when you need help in the first place. Thank you, Ken, for joining us. Really happy to have you on to talk about what you've been working on at First Person Labs. Is this the first time you're talking about it? Yes. Yes. So what do you do? So we focus on essentially creating a model of a person, and then put that person in control. And the model of a person basically means that understanding their memories, understanding what they do, but more of that, their priorities, preferences, goals, values, and commitments, and intent. So I... Sorry, just let's just make this very simple. I'm a big Grock Bot user, so I use GrockBot a lot. Where would First Person Labs fit into my usage of GrockBot? Yeah. The way it works is that you would just bring a context over to GrockBot. It's like portable context. You go to GrockBot and you say, hey, can you talk to my thing to figure out what I want? And then the two of them will start talking, and then you can just send prompts like buy me a flight, and you would know what flight you're …

Get the full transcript (7,134 words) + summary by email — free

One-time email with the complete transcript and AI summary of this episode. No account needed.

One email, no spam. We’ll also show you what SignalCast does.

Browse all a16z Podcast transcripts →

You just read a 3-minute summary of a 30-minute episode.

Get a16z Podcast summarized like this every Monday — plus up to 2 more podcasts, free.

Pick Your Podcasts — Free

Keep Reading

More from a16z Podcast

We summarize every new episode. Want them in your inbox?

Similar Episodes

Related episodes from other podcasts

Explore Related Topics

This podcast is featured in Best Business Podcasts (2026) — ranked and reviewed with AI summaries.

Read this week's Startups & Product Podcast Insights — cross-podcast analysis updated weekly.

You're clearly into a16z Podcast.

Every Monday, we deliver AI summaries of the latest episodes from a16z Podcast and 192+ other podcasts. Free for one show.

Start My Monday Digest

No credit card · Unsubscribe anytime