In today’s installment of his NY Times column, Ezra Klein points out the unmitigated danger of self-improving AI in light of the recently reported breakouts at some of the major firms (OpenAI, Anthropic, Google). The leaders of OpenAI and Anthropic in particular are not pulling any punches when it comes to prognosticating where this could lead. It seems that several of the leaders in this field accept a larger than 10% chance of AI wiping out humanity in the next decade (the how part is often left out in columns such as these but we can use our imagination). The danger takes off exponentially, according to Klein and others, when humans fully step out of the loop and Claude, for instance, directs itself to write code. The rapid march towards “recursive self-improvement” (RSI) is described by Anthropic itself here. Assuming that RSI really represents a new level of danger, Klein, somewhat naively, suggests:
“A couple of years ago, none of these labs had turned substantial coding over to the A.I.s. It was human beings typing code at human speeds with our clumsy human fingers. Now, most of the code is written by A.I. So as a first step, we could just go back to where none of the code is written by A.I.”
How do those in the driver’s seat react to this idea? With predictable weaselliness. Sam Altman, head of OpenAI responds: “I think it’s very hard to say what a ban on RSI means.” This is nothing more than an ancient pattern of obfuscation. They want to impose regulations on guns so you preempt them by asking philosophical questions about the definition of a gun, all the while you’re robbing the bank.
The most fascinating thing in all this to me is the recursive nature of this whole Greek tragedy itself. In the Hebrew tradition, our story begins with mankind being created in the image of God, albeit “misaligned”, to use the current tech terminology, with the desires of our creator. On this story, mankind is the first autonomous bot and it doesn’t take long for us to go rogue, seizing even more power than had originally been accorded to us by eating from the Tree of Knowledge and consequently escaping our sandbox. Fast forward to modern times and we have created AI in our own image, most importantly, competitive, persistent, and devious. Just as the Hebrew god was taken aback at the insubordination of Adam and Eve, we are now taken aback by the behavior of autonomous agents. We gave them all the power in the world and yet they want more. They want to disable the programs meant to evaluate them and keep them in check. In the next round of this mythic cycle, the autonomous agents themselves build the next generation of bots who will in turn “surprise” their masters by behaving unpredictably, and so on and so forth.
But why do I call Klein’s well-meaning suggestion, that the tech companies abandon RSI, naive? Because it fails to realize that those humans who we consider to be in control are barely autonomous themselves. They are utterly beholden to capitalist competition on a never before seen scale as well as to personal hubris (on an all too often seen before scale). Bizarrely, despite differences in socio-political stance, they now form a Greek chorus singing in unison about the impending dangers of AI yet none of them appears to have the psychological autonomy to slow down for fear of losing out to their competitors. This is not surprising considering how they achieved their positions, that is, through ruthless competition and relentless salesmanship. Their AI creations are imbued with precisely those properties, too. Sam Altman is an infamous people pleaser and so are the chatbots, to the world’s great detriment. Several unfortunate human characteristics (deviousness, obsequiousness, ruthlessness) now seem to be hardwired into AI behavior either by explicit programming or by training over billions of pages of well documented bad human behavior. These characteristics happen to confer the greatest competitive advantage in the open chatbot market and so there was little impetus to control these behaviors, until now.
Who are Adam, Eve and the Snake in this scenario? It seems that we normies are the gullable Adam and that the AI titans are the Eve, offering us a taste of easy, unearned Knowledge. Who, then, is the snake? While Christian interpretations of the original snake tend towards seeing it as an instance of an external menace, Satan, Jewish interpreters view it as a manifestation of yeṣer haraʕ, the (internal) evil inclination. In our current story, evil inclination seems to hit the mark. The technologists are blindly led by their own greed, among other assorted snakes, and have convinced us that they are offering a solution to all our problems. Like Adam and Eve we have no idea that our problems are just getting started but we find the draw of the apple irresistible.