Can somebody give a good faith explanation of existential risk posed by AI? I always see MIRI/rationalists talk about how this is so important to prevent, but I don’t think I actually have a good understanding of how an “evil AI that destroys humanity” would actually come about or pose a threat.
Caveat – I very much understand the less dramatic examples of unaligned AI (racial bias, etc.).
You remember I,Robot? (either the book, or the Will Smith movie) The one where they programmed all the robots with three, very sensible safeguards against harming humans, but then they enslaved all humanity anyways, in order to protect humanity from itself?
Basically that. The idea is that a sufficiently smart AI will find highly creative, definitely unethical ways to accomplish the task you gave it. Sorcerer’s Apprentice, monkey’s paw, that basic trope.
So basically, rationalists are very keen on “solving ethics”, so they can give the robots a complete, objective, and unambiguous list of all the things they are not allowed to do.
no need to imagine: it already exists and it’s called the Facebook News Feed
(not even being ironic; whatever you call AI vs. machine learning, the first and possibly last major AIs are going to look like faceless algorithms not sexy chrome androids)
Other comments do a good job of answering your question, but since this is SneerClub here’s a relevant sneer: It’s pretty telling how much more concerned they are over their dramatic hypotheticals than they are the confirmed harmful impact implementations of AI have on minorities today.
Here’s a short version:
The range of possible goals and values a given AI could have is huge, and only a small portion of that leads to a future we’d be happy about if that AI gains incredible power. The major reason for this is that controlling more space, matter and energy is useful in some way to most possible goals it might have. Even if it only wants to compute pi, or make art, turning the entire planet into a giant calculator or mechanized art studio is better for those goals than letting us live on it. The asserted principle is that goals and intelligence are independent - you can be really smart and only care about making towers on the moon, for example. So it’s not so much that the AI might hate humanity as simply valuing something different than what we value, and thus eliminate us and what we care about for a benefit to itself.
Whether this scenario is a significant risk depends on how likely you think it is that an AI could quickly gain superhuman power before we can notice and counter it. This is where the MIRI/rationalist crowd drums up urgency.
A classic https://www.lesswrong.com/tag/paperclip-maximizer
Its also a cause that is dramatic, sexy, and impresses Silicon Valley types with more money than they know what to do with, which enables Yud and Co to get grants to each computers how to run a DnD game.
No, never
from lawful.good import *
Boom, problem solved.
Even simple systems have unintentional properties.
In the Sims 4 pets, they added a Roomba-like thing. You can lock certain rooms so pets can’t get into them. Cats can sit on Roombas. If a cat sits on a Roomba, and the Roomba goes in the room, the cat is able to enter.
Don’t know if you ever played old school dnd, and remember the wish spell, and how a annoying dungeon master can just twist your words forever and ever and always guarantee a bad outcome? (If you don’t know this, the movie wishmaster (this link is reasonably safe, but if you look for more remember it is a horror movie) basically is about this.
MIRI worries that an general AI (AGI, or strong AI, aka an AI which is intelligent can solve general problems and can self improve (it has enough of all of these that it can convince humans to do its bidding and improve on it (please install an additional RAM chip Dave), optionally it is also conscious) will do the same when people turn it on and give it a mission like ‘make paperclips’ (and it then turns the universe into paperclips).
E: certainly not a dumb question btw, it is quite confusing that they are using a different definition of AI than the rest of the world uses.
I like this one: https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html - it’s long but it’s entertaining and has lots of images and some good examples.
And it’s not a dumb question, rats talk about it as though everyone knows that it’s obviously important/etc even though it is a very fringe idea.
https://www.buzzfeednews.com/article/tedchiang/the-real-danger-to-civilization-isnt-ai-its-runaway
One small question: why are you asking this question on here? This is a forum explicitely dedicated to sneering about people who talk a lot about AI existential risk scenarios, and sometimes about their arguments as well.
I think this is the best introductory explanation I’ve seen: https://www.vox.com/future-perfect/2018/12/21/18126576/ai-artificial-intelligence-machine-learning-safety-alignment
I find it very silly because we already have emergent algorithms screwing us (capitalism etc)