Ep. 377: The Case Against Superintelligence

A couple weeks ago, the techno philosopher and AI critic, Eleazar Yatkowski, went on Ezra Klein's podcast. Their episode had a cheery title, How Afraid of the AI Apocalypse Should We Be? Yudkowsky, who recently coauthored a book titled, If Anyone Builds It, Everyone Dies, has been warning about the dangers of rogue AI since the early two thousands. But it's been in the last half decade, as AI began to advance more quickly, that Yudkowsky's warnings are now being taken more seriously. This is why Ezra Klein had him on. I mean, if you're worried about AI taking over the world, Yudkowsky is one of the people you want to talk to. Think of him as offering the case for the worst case scenario. So I decided I would listen to this interview too. Did Joukowsky end up convincing me that my fear of extinction should be raised? That AI was on a path to killing us all? Well, the short answer is no. Not at all. And today, I want to show you why. We'll break down Yekowsky's arguments into their key points, and then we'll respond to them one by one. So if you've been worried about recent chatter about AI taking over the world, or if like me, you've grown frustrated by these sort of fast and loose prophecies of the apocalypse, then this episode is for you. As always, I'm Cal Newport, and this is Deep Questions. Today's episode, the case against superintelligence. Alright. So what I wanna do here is I wanna go pretty carefully through the conversation that Jadkowski had with Cline. I actually have a series of audio clips so we can hear them in their own words, making what I think to be are the the the key points of the entire interview. Once we've done that, we've established Joukowsky's argument, then we'll begin responding. I would say most of the first part of the conversation that Joukowsky had with Klein focused on one observation in particular, that the AI that exists today, which is relatively simple compared to the superintelligences that he's worried about, even today in its relatively simple form, we find AI to be hard to control. Alright. So, Jesse, I want you to play our first clip. This is Jachowsky, talking about this phenomenon. So, there was a case reported in, I think, the New York Times where a kid had a like, a 16 year old kid had a extended conversation about his suicide plans with Chatt GPT. And at one point, he says, Should I leave the noose where somebody might spot it? And ChatGPT is like, No. Let's keep this space between us, the first place that anyone finds out. And no programmer chose for that to happen is the consequence of all the automatic number tweaking. Alright. Let's cut it off there, Jesse. Alright. So to Yacowski, this is a big deal that no programmer chose for, say, ChatCPT to give advice about …