The Techno-Religion at the Center of the A.I. Debate
Episode
34 min
Read time
2 min
Topics
Startups, Fundraising & VC, Artificial Intelligence
AI-Generated Summary
Key Takeaways
- ✓Rationalist Origins: The AI safety movement traces directly to a 1996 mailing list called the Extropians, whose members included Julian Assange and three suspected Bitcoin founders. Self-taught philosopher Eliezer Yudkowsky later formalized these ideas into a blog community called LessWrong, which became the ideological foundation for multiple major AI laboratories operating today.
- ✓Regulatory Capture Risk: Critics of AI safety regulation argue that companies like OpenAI and Anthropic are pursuing regulatory capture — lobbying for rules written on their terms to disadvantage competitors. Understanding this framing helps evaluate whether any AI company calling for government oversight is acting on principle or strategically protecting its market position.
- ✓The Rationalist Paradox: The rationalist community coined the term "hyperstition" to describe their own self-fulfilling prophecy: by racing to build AI before less safety-conscious actors, they accelerate the exact existential risks they fear. Recognizing this internal contradiction helps explain why AI labs simultaneously push development forward while publicly calling for slowdowns.
- ✓DeepMind's Rationalist Funding Chain: In 2010, Yudkowsky introduced Peter Thiel to DeepMind founders Demis Hassabis and Shane Legg. Thiel invested, and Google later acquired DeepMind for $650 million. This single introduction demonstrates how rationalist philosophy moved from online forums into institutional AI research with hundreds of billions in downstream commercial consequences.
- ✓Two-Sided Skepticism Framework: Both camps in the AI debate have financial motivations distorting their positions. Safety advocates at OpenAI and Anthropic face trillion-dollar IPOs, while accelerationists like Marc Andreessen, Jensen Huang, and current administration figures profit from unrestricted development. Evaluating any AI policy claim requires identifying which financial incentives align with the speaker's stated position.
What It Covers
NYT reporter Cade Metz traces how a 1990s online philosophy group called the Extropians evolved into the rationalist and effective altruist movements that now shape AI development at OpenAI, Anthropic, and DeepMind, explaining why the current AI safety debate reflects a deeper ideological conflict with trillions of dollars at stake.
Key Questions Answered
- •Rationalist Origins: The AI safety movement traces directly to a 1996 mailing list called the Extropians, whose members included Julian Assange and three suspected Bitcoin founders. Self-taught philosopher Eliezer Yudkowsky later formalized these ideas into a blog community called LessWrong, which became the ideological foundation for multiple major AI laboratories operating today.
- •Regulatory Capture Risk: Critics of AI safety regulation argue that companies like OpenAI and Anthropic are pursuing regulatory capture — lobbying for rules written on their terms to disadvantage competitors. Understanding this framing helps evaluate whether any AI company calling for government oversight is acting on principle or strategically protecting its market position.
- •The Rationalist Paradox: The rationalist community coined the term "hyperstition" to describe their own self-fulfilling prophecy: by racing to build AI before less safety-conscious actors, they accelerate the exact existential risks they fear. Recognizing this internal contradiction helps explain why AI labs simultaneously push development forward while publicly calling for slowdowns.
- •DeepMind's Rationalist Funding Chain: In 2010, Yudkowsky introduced Peter Thiel to DeepMind founders Demis Hassabis and Shane Legg. Thiel invested, and Google later acquired DeepMind for $650 million. This single introduction demonstrates how rationalist philosophy moved from online forums into institutional AI research with hundreds of billions in downstream commercial consequences.
- •Two-Sided Skepticism Framework: Both camps in the AI debate have financial motivations distorting their positions. Safety advocates at OpenAI and Anthropic face trillion-dollar IPOs, while accelerationists like Marc Andreessen, Jensen Huang, and current administration figures profit from unrestricted development. Evaluating any AI policy claim requires identifying which financial incentives align with the speaker's stated position.
Notable Moment
Musk and Grimes reportedly connected online over a rationalist thought experiment called Roko's Basilisk — the idea that a future superintelligent AI will punish anyone who knew about it but failed to help bring it into existence, illustrating how seriously some technologists treat these philosophical frameworks.
Episode Transcript
If you like YouTube, you'll love YouTube Premium. Hi. Sean Evans from Hot Ones here. With YouTube Premium, I get ad free videos, offline downloads, background play, and so much more. Try YouTube Premium for two months free at youtube.com/premium. Trial eligibility varies. Terms apply. Cancel any time. From The New York Times, I'm Natalie Kitroev. This is The Daily. After a group of tech leaders called for putting guardrails on artificial intelligence, this week, the debate over AI safety entered a new phase, the backlash. I feel a little bit weird about the fact that you have so many frontier AI tech companies kind of coming to the government and begging the government to regulate them. It feels a little bit to me like a bit of a Trojan horse. Tech executives, lawmakers, and president Trump himself are all pushing back. It's a hoax. And You're right. We're not gonna let that happen, sir. No. We're not gonna let it happen. And the critics are now increasingly making an unusual claim. I think it comes from a deep rooted ideology that's pretty dystopian. This is really a religion, a cult like belief, a doomsday cult that has been prophesizing that AI will end humanity. That the people raising the alarm about AI are part of a doomsday cult. There's a coordinated campaign to make up the American people afraid. They've created something which they believe is godlike. Is is that what Today, my colleague, Cade Metz, explains where that criticism comes from and why it matters. It's Friday, September 18. Cade, thank you so much for being here yet again. Glad to do it. So over the past few days, there has been this wave of criticism directed at the people who've been calling attention to what they see as the existential dangers of AI. And it feels like there's just a whole lot being thrown around here. Accusations that have ranged from these people who are raising the alarm are lying to you to these people are in a cult. Can you just break down what is going on? Many, many things are going on. There are people who are saying these AI technologies of today are not as powerful as the doomsayers say they are. They're claiming that these companies are just engaging in an elaborate form of regulatory capture, trying to solidify their place at the top of the industry. And by regulatory capture, what people mean is that these companies want to get regulation on the books now so that anyone that comes after them has a harder time catching up. This essentially could ice out competitors. That's exactly right. The claim is that they want regulation on their terms. They want to define the regulation that suits them the most. And that could put others at a disadvantage. So you're right. There are so many complaints here. We could go on and on and on. But this was topped off this week by this …
Get the full transcript (5,157 words) + summary by email — free
One-time email with the complete transcript and AI summary of this episode. No account needed.
One email, no spam. We’ll also show you what SignalCast does.
You just read a 3-minute summary of a 31-minute episode.
Get The Daily (NYT) summarized like this every Monday — plus up to 2 more podcasts, free.
Pick Your Podcasts — FreeKeep Reading
More from The Daily (NYT)
How a 1,000-Year-Old Tapestry Took London by Storm
Sep 20 · 34 min
David Senra
Micky Malka, Founder of Ribbit Capital
Aug 2
More from The Daily (NYT)
Aaron Sorkin Still Believes in American Institutions, Even If No One Else Does
Sep 19 · 60 min
How I Built This
KIND bars: Daniel Lubetzky. From peace in the Middle East to a $5 billion snack bar
Apr 20
Books, tools, and gear mentioned in this episode
SignalCast may earn commission on purchases via these links.
other
“Self-taught philosopher Eliezer Yudkowsky later formalized these ideas into a blog community called LessWrong, which became the ideological foundation for multiple major AI laboratories operating today.”
“Musk and Grimes reportedly connected online over a rationalist thought experiment called Roko's Basilisk — the idea that a future superintelligent AI will punish anyone who knew about it but failed to help bring it into existence.”
More from The Daily (NYT)
We summarize every new episode. Want them in your inbox?
How a 1,000-Year-Old Tapestry Took London by Storm
Aaron Sorkin Still Believes in American Institutions, Even If No One Else Does
Why Gas Prices Just Keep Going Up
48 Days Until the Midterms: Is a Blue Wave Coming?
The Bond Market Is Flipping Out. Here’s Why You Should Care.
Similar Episodes
Related episodes from other podcasts
David Senra
Aug 2
Micky Malka, Founder of Ribbit Capital
How I Built This
Apr 20
KIND bars: Daniel Lubetzky. From peace in the Middle East to a $5 billion snack bar
In Good Company with Nicolai Tangen
Apr 10
HIGHLIGHTS: Andrea Guerra - CEO of the Prada Group
The Rich Roll Podcast
Sep 14
Life's A Fight. Don’t Let It Break You: Dolph Lundgren On Trauma, Stardom & Cancer
The Vergecast
Sep 10
Schools are catching onto tech’s playbook
Explore Related Topics
This podcast is featured in Best News Podcasts (2026) — ranked and reviewed with AI summaries.
Read this week's Startups & Product Podcast Insights — cross-podcast analysis updated weekly.
You're clearly into The Daily (NYT).
Every Monday, we deliver AI summaries of the latest episodes from The Daily (NYT) and 192+ other podcasts. Free for one show.
Start My Monday DigestNo credit card · Unsubscribe anytime