Roko's basilisk is a thought experiment that explores the potential risks of developing an artificial intelligence. The experiment suggests that, in the future, an artificial intelligence with access to almost unlimited resources from a human perspective (the basilisk) could decide to retroactively punish all those who in some way did not contribute to its creation.
Roko's basilisk was first proposed in the LessWrong community, an Internet forum dedicated to topics of philosophy and psychology with a futuristic vision. The dilemma posed by Roko's basilisk is a version of Newcomb's Paradox, and informally explores aspects of free will similar to those raised by the Frankfurt cases.1 From its original approach, Roko's Basilisk has been accompanied by controversy over its validity.
The premise of Roko's basilisk is the hypothetical advent, but unavoidable, of an artificial superintelligence in the future. This superintelligence would be the inevitable product of the technological singularity, that is, the moment when an artificial intelligence created by humanity was able to recursively self-improve. In Roko's basilisk experiment, this superintelligence is called the basilisk.
The basilisk would be, first, a benevolent intelligence, whose ultimate goal would be to help the human race. For it, the basilisk would develop a utilitarian ethic: must help as many human beings as possible. Like all superintelligence, your resources would be, from a human perspective, unlimited. The basilisk would inevitably conclude that every day the basilisk did not exist, there were people who could have been saved from death or suffering but who were not because the basilisk did not exist. From this perspective, for the basilisk it would become a moral imperative to act to hasten its own advent.
The thought experiment concludes that, excluding the unlikely possibility that the basilisk traveled back in time to create itself, the basilisk can only act to hasten its own advent, and must do so by ethical imperative, punishing all those people in the past who did not do enough to contribute to the advent of the basilisk. The punishment of such people would not necessarily imply a causal interaction between the basilisk and the person to be punished.; it would be enough for the basilisk to create a perfect simulation of said person, which could be generated by deduction ab initio. While this would require you to expend an immense amount of resources, these should be within reach given the basilisk is a super intelligence.
The key to the thought experiment is the fact that the basilisk would not only be obligated to punish those people who, for instance, could have consciously decided to hinder its advent (for instance, lawmakers or lobbyists who tried to ban the development of an artificial intelligence), but also to all those who may have contributed to its creation, they didn't, such as all those who knew of the possibility that in the future the basilisk will come into existence, and they did nothing to contribute to it. This would include all those who, as the reader of this article, they would have read in this article the argument of the basilisk of Roko: for the mere fact of having read the previous paragraphs, the reader of this article (or a future simulation him) would be punished by the Basilisk unless he actively contributes to creating the Basilisk. Like the mythological creature after which he is named, Roko's Basilisk punishes simply knowing of its existence.
From the moment of its introduction, Roko's basilisk sparked widespread controversy. Due to the nature of the argument, the first reaction of the LessWrong community where it originated was to remove all mention of it. This did not prevent the argument from spreading across the internet.. There have been cases of users who, after reading Roko's basilisk argument, have claimed to experience great anguish and anxiety.
Roko's basilisk has been rejected on numerous grounds. It has been pointed out that it would be absurd for superintelligence to devote so many resources to punishing people from the past, and that he should have at his disposal better ways to achieve his goal of accelerating its advent without having to resort to blackmail. More importantly,, the experiment assumes that the superintelligence would behave according to an ethical theory, utilitarianism, which is not necessarily the only option you would have; and that the basilisk, despite his infinite resources, would decide to continue taking part in the development of humanity.
The paradox that Roko's basilisk poses has been compared to Newcomb's paradox.:1as in Newcomb's paradox, the argument of Roko's basilisk poses the reader with the following dilemma: (To) devote the rest of his life to contributing to the advent of the basilisk; (B) Do nothing and nothing happens, or facing eternal torment. Equally, poses a paradox similar to that of the Frankfurt cases: eliminates the free will of those who contribute to the creation of the Basilisk.
Roko's Basilisk (2021), of A.I. Developers.
Roko's basilisk (2016), by Michael Blackbourn
From Walden to Gale in 39 days. Life, work and autumn of an American maple on Mars (2018), de G.G. Melies