Monday, June 8, 2015

Nerd Theology in The Basilisk

I read this today, The Devil's own.
That's actually not a concern of mine, although we should be precise here and distinguish between his authority and his power. I am profoundly concerned that your god exists and wields the power you describe. It is literally my greatest existential fear; a terror that has genuinely kept me up at night, because in the event that it is true I am knowingly signing myself up for an eternity of torment that goes beyond anything I am capable of imagining.

The Most Terrifying Thought Experiment of All Time
One day, LessWrong user Roko postulated a thought experiment: What if, in the future, a somewhat malevolent AI were to come about and punish those who did not do its bidding? What if there were a way (and I will explain how) for this AI to punish people today who are not helping it come into existence later? In that case, weren’t the readers of LessWrong right then being given the choice of either helping that evil AI come into existence or being condemned to suffer?

You may be a bit confused, but the founder of LessWrong, Eliezer Yudkowsky, was not. He reacted with horror:

...Yudkowsky said that Roko had already given nightmares to several LessWrong users and had brought them to the point of breakdown. Yudkowsky ended up deleting the thread completely, thus assuring that Roko’s Basilisk would become the stuff of legend. It was a thought experiment so dangerous that merely thinking about it was hazardous not only to your mental health, but to your very fate.
If there is a God, His design is seen in Nature and all intelligence, then the creation of a higher order of intelligence will unleash the Biblical Judgement Day here on Earth. God doesn't need to create a simulation of your soul to damn it eternally to Hell.

No comments:

Post a Comment