Are We About to Lose Control of AI? | AI Reality Check

Last week, Anthropic released a report with a scary sounding title when AI Builds Itself and it came accompanied by a scary animation that shows machines replicating themselves exponentially like cells in a petri dish. Now, the body of the report itself keeps these dark vibes going. I want to read you some actual quotes here from the intro to the report. They say For most of AI's history, humans drove every step in its development cycle, but at Anthropic, we are delegating a growing share of AI development to AI systems themselves, which is speeding up our work.

Taken far enough and given enough compute, this trend points to an AI system capable of fully autonomously designing and developing its own successor. This is called recursive self-improvement. We are not there yet, and recursive self-improvement is not inevitable, but it could come sooner than most institutions are prepared for. A little bit later, they then add, AI that can build itself would be a major development in the history of technology, one that could bring enormous good for the world in science, healthcare, and beyond, but full recursive self-improvement also might increase the risks of human humans losing control. over AI systems.

Now if you look at the the headlines generated in response to this report, most of them focused on a section of the report that uh seemed to call for a worldwide pause on AI development to avoid the scenario of humans losing control. But if you read that section closer, you see that's not actually what the report says. Here's the actual wording. If it were possible to effectively slow the development of this technology to give ourselves more time to deal with its immense implications, we think that would likely be a good thing. But if a slowdown simply lets the least cautious actors catch up technologically, it could leave everyone less safe.

We already have this episode, so unlocking it is cheap and permanent: the full transcript (21 minutes, 3,690 words), segment and word-level timestamps, and .txt, .srt, .vtt and word-level JSON downloads, as many times as you like, forever.

No account needed, we create one from your checkout email. With credits it is 1 credit; packs start at 10 episodes for $5, and monthly plans are cheaper still.