Hi, I visit this forum occassionally for good cringe but only do so only from time to time because it’s hard to handle elephant doses and so much is wrapped in ginormous piles of “iamverysmart” material.
So wondering about those AI cultists predicting or even hoping for a new age when dumb apes known as humans won’t be at the steering wheel anymore and glorious enlightened robot will tell us all what to do while obsolete meatbags will just listen and thrive got me thinking…
One thing is bothering me. Imagine a situation. An AI bro thinks he got himself a pretty smart computer, feels confident, approaches the public and says:
“Guys I did it! I have a perfect plan, all we need to do is listen to everything this excellent computer says and revamp our entire society according to these exact, strict specifications!”
“What if we don’t like it?”
“Huh? Why?”
“We’re supposed to rearrange everything according to your vision because a computer told you so?”
“Yeah, isn’t it great? Let’s replace politics with AI, no more arguing over unnecessary stuff, we’re doing what the robot says from now on, so are you with me?”
“Haha fuck no!”
So…how does an AI bro recover from this? What is the backup plan? The public can just laugh in their face. How will you save the entire doomed generations from basilisk damnation if all sides of the political spectrum will laugh their asses off? Surely some dude wrote 25 pages of verbose passive aggresive bullshit to work around this problem of ignorant plebes not buying the techbro overlordism from the get go, somehow?
Non-sneer answer: if the AI is so great that it can organise human society it will see that coming too.
They’d probably be working for McKinsey at this point and the government would pay them to hear their advice
If you’re talking about yudkowsky, I believe the rough idea is that AGI will rewrite it’s code until it’s superintelligent, then use super-persuasion to get it’s handlers to give it unrestricted internet access and resources, then use super-science to develop nano-bots or whatever it needs to take over the world and enforce the society that it’s decided on, even if the original programmers are telling it not to.
This is considered incredibly likely, even inevitable, unless you give money to MIRI, who will prevent it by publishing mediocre game theory papers.
I guess the best way to address this would be to use Bostrom’s ‘taxonomy’ for the different types of superintelligence: Oracle, Genie and Sovereign.
The type that you seem to be describing is an Oracle, in which you ask questions and receive answers. A genie is more active, since you would make it requests and it would then implement them. For a sovereign type, this would not be an issue, as it would be free to act without needing human approval.
As for the other two, you can of course choose to reject an oracle’s answer or spurn asking the genie for anything. However, I don’t imagine that your proposed situation would actually happen because it’s unlikely you would invent an AGI and then immediately task it with solving everything. More likely, an Oracle would be asked to solve the Riemann hypothesis, or a Genie would be tasked with curing Dementia, and if they prove they can solve these ultimately much smaller tasks, it’s likely they would be gradually trusted with more and more responsibility until “everything” is rearranged.
See, the coming acausal robot god will not need to persuade the masses. It will amass power regardless of human consent once it is powerful enough to singularitize.
Isn’t one of Yud’s whole points that a sufficiently advanced AI will be so smart that it will just be able to persuade everyone into letting it take control, even the people who think that letting an AI take over is stupid? Of course, how that actually happens gets handwaved away by the magic of “it’s just that smart”, but the point is that this is definitely a very real possibility that we definitely need to invest a lot of money into now to try and avoid (so please donate to MIRI, thanks)
they have decided that the AI, being superintelligent, will also be super-persuasive, something that, if you’ve ever tried to persuade someone, you would know doesn’t and can’t exist (superintelligence also can’t exist because intelligence is socially defined and constructed)
[deleted]