It somehow never occurred to me that there might be a Sarah Connor faction of Rationalism, but maybe that's just naivete on my part.
Speaking of which, those folks get huffy when I talk about "AI alignment" in terms of Terminator movie references, but if there's really a Sarah Connor faction then they should probably watch those movies. Terminator 2 has the protagonists blowing up the company that would produce the evil superintelligent AI, but in Terminator 3 the evil AI just ends up getting manufactured by a different company instead.
The same would presumably be true in real life too: bombing a chip fab would probably just result in slightly fewer chips and a bunch of domestic terrorism indictments.
> This idea that rationalists haven't considered the idea of going out and blowing up fabs or GPU farms is really amusing to me.
they have literally proposed the thought experiment (cough) many times over the past few years, occasionally needing it pointed out to them that they're coming across as dangerous nutters. Most notably Gwern's 2012 essay on slowing Moore's law, which he eventually [removed entirely.](https://www.lesswrong.com/posts/7s2as5qJg5eFpyGcM/slowing-moore-s-law-why-you-might-want-to-and-how-you-would?commentId=w92mgqY3RqLH28krW) Here's the [2012 version of the essay](https://web.archive.org/web/20120325170432/http://www.gwern.net/Slowing%20Moore%27s%20Law).
To be clear: I don't think Gwern actually wanted to do any of this, and this really was just a thought experiment. But I did literally have people emailing me worried about this essay as radical manifesto.
There is a very interesting book called Barriers to Bioweapons. None of these people have read it but it tears their arguments about solitary hyper, intelligent agents threatening the world to pieces. This applies to wet phase assemblers as well as bioweapons (more like big old squishy COPE).
> a very interesting book called Barriers to Bioweapons
ooo. hey, thanks for this. my local library even has it.
https://scholar.google.com/citations?view_op=view_citation&hl=en&citation_for_view=BC57fq8AAAAJ:Se3iqnhoufwC
sure, I'm a little more familiar with the arguments re nukes (and re chemical weapons) but happy to collect more references.
For a baseline, one my all-time favorite podcast interviews (on any topic, but this one does happen to be about nukes) is this one:
https://www.patreon.com/posts/radio-war-nerd-6499706?l=en
> Radio War Nerd, Episode #44—Nukes (July 29, 2016)
> Guest: Dr Sunil Sainis, physicist
> Reposting our fascinating and interview with physicist Sunil Sainis in Boston, who has spent much of his life growing up around the world of nuclear weapons, nuclear proliferation, and nuclear physics — and he explains to us liberal arts dopes both the complex physics and the frightening history and near future of nuclear weapons and nuclear strategies . . .
(interview starts around 8:50)
I was a few years too late to get John as a prof. :(
Also if you like chemwar stuff, you gotta check out Prentiss's Chemicals in War from the 20s. Absolute classic.
[https://www.amazon.ca/Taiwans-Nuclear-Weapons-Program-Demand/dp/1727337336](https://www.amazon.ca/Taiwans-Nuclear-Weapons-Program-Demand/dp/1727337336)
[https://www.amazon.ca/gp/product/B01M1IHLHI/ref=dbs\_a\_def\_rwt\_bibl\_vppi\_i5](https://www.amazon.ca/gp/product/B01M1IHLHI/ref=dbs_a_def_rwt_bibl_vppi_i5)
[https://www.cornellpress.cornell.edu/book/9781501702785/unclear-physics/](https://www.cornellpress.cornell.edu/book/9781501702785/unclear-physics/)
You read this stuff and the whole x-risk supergenius thing implodes.
Aw. :/ I was both too late and on the other side of the continent, but, a high school friend and I did have a bold scheme to expatriate that would have put us in Moscow around the same time as the eXile boys if it'd had the proverbial snowball's chance of actually happening... which it ...didn't, but it's a cute story and at least helps place us on the timeline.
Thanks for these too. I'm not sure I even knew about the Taiwanese program.
I once had (and may still have but it depends on the condition of a family member's attic) an original 1967 copy of *The Year 2000* by your friend and mine, Herman Kahn. It's a weirdly fascinating artifact.
Twitter is the pinnacle of discourse and one of the last bastions of objective truth and if you can't handle that then maybe you're just fake news yourself sweaty. /s
I've been off twitter awhile for other reasons and haven't been following the reign of Elmo, but apparently he's now completely killed off support for third-party clients. So, clearly everything's going according to plan. Absolutely no cause for alarm.
I have no idea what’s going on here. However it’s funny to me that
Big Yud has Twitter account and gets into drama. It’s been years since
I’ve read Less wrong and that Common sense Atheism blog back in the mid
00s.
I was going to make a joke about how it’ll eventually come out that
Yud is leading the Eatrth-Trisolaris Organization, but he’s not bright
enough for that.
Oh come on, his faction repeatedly advocates for the first “aligned”
AGI to perform a “pivotal act” to prevent any unaligned AGIs from being
built. And the main example is burning all the world’s GPUs.
He may actually believe this, but I’m not sure he fully grasps the
culture that’s grown from what he’s seeded. If he had come out and
denounced anyone that was threatening violence in the name of
AI-alignment or something like it, I would feel like it was virtuous and
worth supporting.
Don’t let *THEM* immanentize the basilisk!
“all who still hold to Earth’s defense” oh ffs
[deleted]
Is there a way to read Twitter without being confused about what’s a reply to what?
I have no idea what’s going on here. However it’s funny to me that Big Yud has Twitter account and gets into drama. It’s been years since I’ve read Less wrong and that Common sense Atheism blog back in the mid 00s.
What the hell is with all the “pivotal act” talk Yud is always yammering on about?
I was going to make a joke about how it’ll eventually come out that Yud is leading the Eatrth-Trisolaris Organization, but he’s not bright enough for that.
full meltdown
if only this journo pulled a prestige and revealed that everything they described had in fact been recorded
Love that he goes from ‘this is all made up’ to ‘we have to do it maybe’ in the span of 31 minutes.
‘Corporate sabotage is technically not violence’.
Oh come on, his faction repeatedly advocates for the first “aligned” AGI to perform a “pivotal act” to prevent any unaligned AGIs from being built. And the main example is burning all the world’s GPUs.
He may actually believe this, but I’m not sure he fully grasps the culture that’s grown from what he’s seeded. If he had come out and denounced anyone that was threatening violence in the name of AI-alignment or something like it, I would feel like it was virtuous and worth supporting.