Eh. While his reasoning is fairly scattered here “I’m not actually terrified of AI coming to life and eating us” is not a hot take most places.

I know mathematicians in the field who are genuinely annoyed it’s a hot topic now. They’re more worried about their work aiding with the whole “embedding shitty societal biases in black box systems and then keeping said biases forever, actually entrenching us deeper in inequality rather than uplifting anyone except the mega-wealthy” thing than “robot god”. I’m not saying “robot god” is impossible but it’ll look a hell of a lot more like Minority Report and a hell of a lot less like Terminator.

Which he seems to agree with with:

I think it’s entirely plausible that, even as AI transforms civilization, it will do so in the form of tools and services that can no more plot to annihilate us than can Windows 11 or the Google search bar. In that scenario, the young field of AI safety will still be extremely important, but it will be broadly continuous with aviation safety and nuclear safety and cybersecurity and so on, rather than being a desperate losing war against an incipient godlike alien.

But yes, cringe at how this (like all things, if you listen to bloggers) is kind of like nazis, and ‘oh no i was too smart as a child, poor me’ (I mean come on dude)

Scott I am begging you, since you are known to check up on us here sometimes, please just stick to blogging about quantum information, you’re actually really good at that and it’s interesting to read. Leave the persecution complex stuff for your therapist please

We should not be tempted to criticize Scott Aaronson’s persecution complex, for that would be judging someone who seems to be in the thrall of a multi-decade mental health crisis, which would be punching down. There but for the grace of god go all of us.

We should instead criticize Scott Aaronson for embracing the Rationalists and their worldview. Aaronson of all people knows that the robot god does not exist, but he gives credence to charlatans like Yudkowsky anyway because he identifies with them emotionally and he likes the validation that he gets from being a part of their community. He is selling out himself and his principles for social approbation.

That’s all pretty consistent with the picture of Rationalism still being in the “cult” stage of religious evolution, though. Cults attract people who are emotionally vulnerable irrespective of their intellectual abilities, and I guess Rationalism is no exception. I think Aaronson deserves better than to settle for being a Rationalist but Aaronson clearly doesn’t share my high opinion of himself.

What is it with these people, who should know better than anyone that something like they describe a superintelligence to be would not remotely resemble a human being, anthropomorphizing it to such a degree. Like, no, an AI is not going to empathize with your fucking high school experience, you utter clod.

Rationalism is largely just nerd identity politics. They’ve made being nerdy the core of their identity and the lens through which they interpret every political issue.

I think the fact that they compare society to a giant high school is telling (Scott Alexander has done it too). They come across to me as having a chip on their shoulder for not being popular in high school and never having got over it.

Some men would literally rather write pages on AI theory than go to therapy.

NERD OPPRESSION

How does anyone bring themselves to actually read posts this verbose. Part of me wants to write a long less wrong post extolling the virtues of brevity.

Vibe: https://youtu.be/L_nHCNz5Mqk?t=228