LessWrong post: My Objections to “We’re All Gonna Die with Eliezer Yudkowsky”
In response to Yudkowsky’s recent podcast interview about how the robot apocalypse is nigh, one brave soul has barged into LessWrong and done the unthinkable: patiently explain, in detail and citing relevant sources from the established academic literature, why Yudkowsky is wrong about everything.
This is LessWrong, so the post still says some oddball things (e.g. giving a 5% estimated chance of robot apocalypse), but on the whole it is a mostly reasonable and sober take on the current state of affairs in machine learning vis-a-vis the plausibility of the end times. I have never seen something like this before on LessWrong.
Some select quotable moments from the post:
How can we stop AI from being evil? OP has a creative and bold suggestion:
As far as I can tell, the answer is: don’t reward your AIs for taking bad actions
Does preventing AI from being evil require the kind of visionary genius that only Yudkowsky can offer? OP suspects not:
I think such issues largely fall under “ordinary engineering challenges”, not “we made too many capabilities advances, and now all our alignment techniques are totally useless”.
OP commits heresy against the acausal robot god:
The misaligned AI does not reach out from the space of possible failures and turn current alignment research adversarial.
OP doesn’t care for Yudkowsky’s tone:
I find [Yudkowsky’s] implication, that I’m only optimistic because I’m inexperienced, pretty patronizing. Of course, that’s not to say he’s wrong, only that he’s annoying. However, I also think he’s wrong.
A general condemnation of Yudkowsky’s opinion on “aligning” AI:
I’m not enthusiastic about a perspective which is so totally inappropriate for guiding value formation in the one example of powerful, agentic general intelligence we know about [i.e. humans].
I don’t necessarily recommend reading the whole post - it’s a bit overwrought - but unlike the average LW post it is not filled with egregious errors or grossly misinformed nonsense.
Recently freed from the shackles of gainful employment, Yudkowsky has time to spare and deigns to share his perspective in some comments.
He begins his defense of himself by noting of the original post that
This is kinda long.
Those, I shit you not, are the very first words that he writes.
He then wonders
why you’d reply to the obviously simplified presentation from an off-the-cuff podcast rather than the more detailed arguments elsewhere.
in the very same comment. Other LWers note the, ahem, apparent discrepancy:
Yudkowsky ultimately responds to exactly one point from the post, saying that the post misunderstood what he said and offering a correction. OP responds in turn; apparently he misunderstood Yudkowsky because Yud’s intended meaning is so brazenly stupid that it never occurred to OP to interpret him in that way. OP then has to update the post with an entire new subsection explaining why Yud is even more wrong than he thought.
Elsewhere, Yudkowsky complains that OP responded to a mere youtube interview, and not to his supposedly better thought out essay “A List of Lethalities”. OP points out that, actually, he already did exactly that in a previous LW post. Eliezer doesn’t respond.
At one point in the post OP asks, rhetorically,
have you ever, even once in your life, thought anything remotely like “I really like being able to predict the near-future content of my visual field. I should just sit in a dark room to maximize my visual cortex’s predictive accuracy.”?
This being LessWrong, multiple commenters say that, actually, yes they have. (comment1, comment2)
In response to a complaint that he’s oversimplifying Yudkowsky’s ideas, OP laments that his efforts at a fuller explanation were stymied by the mind-boggling scope of conspiracy theory whackamole:
there are so many arguments to cover, and it probably would have been ~3 or more times longer than this post
A reminder about Yudokowsky’s total failure to predict the importance of artificial neural networks evolves into a debate about the invention of the airplane, in which we are further treated to corrections regarding Yudkowsky’s overconfident bullshit about the history of aviation.
[deleted]
[deleted]
Mood.
I find it hilarious how even the slightest bit of pushback shatters his whole facade
Amazing for a guy who wants to be taken seriously as the ai alignment guy.
This is what actual research often is Yud, long posts. Not weird tweets. (And I know he likes to play semantics and go ‘I never called myself an ai researcher’, but clearly he wants to be taken seriously).
E: also fun to see how much LW is just weird incrowd jargon. Wonder how much of that is to hide they are not seriously talking about ai alignment, or in LW words how much it is an intentional obfuscation attempt by poisoning the memeplex. (Behaviour which is of course part of the dying wizard sneerclub meme)
overall a fantastic post, and the response to it definitely gets my sneer going
though,
this part does fall kind of flat for me. OOP doesn’t seem to have, at time of writing, thought much about how this example works in a world where people can get overwhelmed by their senses e.g. from autism
and after further thought, OOP seems to agree, and bring in the needed nuance:
Top class sneer.
It will be memory-holed, getting scared about sci-fi is more fun than whatever this is.
The failures of AGI is going to be like global warming, not “alignment” of the AGI itself. That is, the purposes humans put them to, which is a non-trivial problem to solve, but one which EY’s alignment completely fails to address.
🍿
Can someone go respond to the “this is kind of long” comment and just ask him if he has read the sequences?
Glorious. Simply glorious.
lol
I think the funniest thing about all of this is that AI alignment and safety as it’s currently practiced in the ML community is more like nurturing a child than a branch of hard engineering and so of course rationalists are always gonna fail spectacularly at understanding it because the idea of “Teach the machine empathy” just never occurs to them because they’re terminal math fetishists.
How long? Like more than 2 sargons?
Huh? I thought he had never held down a conventional job, but also had patrons who would pay him to post whatever whenever…what’s he lost?
IIRC this is pretty inline with what the average AI researcher thinks.
I used to respect El as an autodidact who had built a name for himself in the AI community and who cofounded LessWrong and MIRI, but the latest fiasco has hurt his credibility. I just hope that he refrains from making such over-the-top claims in the future.
I was fr rolling in immense delight reading this glorious obliteration of Yud