edit: He starts talking about 4chan greentexts less than 2 minutes
in. He really does not give a single fuck about optics.
edit:I listened a bit more. Eliezer is pretending that he has a track
record of underestimating what neural networks are capable of? He
predicted the singularity would happen in 2021!
There was a brilliant comment buried below the video that went something like:
"Wouldn't it be funny if civilization ends because the one person speaking the truth is a fedora'd neckbeard and no one listened [for that reason]?"
And while I don't ever enjoy engaging in ad hominems, for someone who is *supposed* to be brilliant, you would think that it would not take that much mental acuity for him to realize the way he presents himself is so damning to his mission.
I've been wading through too much LW bullshit recently, but there was another relevant quote about the brain-fedora axis I came across:
"Where do you think charisma comes from? The kidney?"
It's almost like he's trying to impress us with his unthinking.
Yeah, that's kind of how I feel about it. If you are are broadly concerned about the future that AI will create, people like Yudkowsky are more of a liability than an asset precisely because he's so transparently a nerd fantasist who's obsessed with outlandish sci-fi scenarios and thus makes such concerns seem ridiculous.
I just want somebody to go, after Yud explained agi and his idea of intelligence as a superpower and acausal trades, go 'wait, isnt this the plot of the hitchhikers guide to the galaxy?'.
I’ve watched about half of this (because I hate myself) and all I can
say is that I hope this serves as evidence for what deeply unserious
thinkers these guys are. But unfortunately I doubt it will have that
effect for most people watching.
He's been working on this for decades. I get the point of sneer club but it's wild to me how confident everyone here is that he's wrong. I guess if he's right, none of us will be here to deal with the consequences, so it's a low risk bet.
You're missing the central point.
It isn't that he's wrong in general. This isn't a place where we deny the possible risks of strong AI, really.
It's that he's a egotistical hack with ridiculous beliefs, *especially* about his professed topic of focus, and a deeply unwarranted sense of self-importance. He's a clown who wants you to take him seriously, like so many self-styled "rationalists".
That kind of stuff doesn't bother me I guess, idk. A lot of people have egos. He seems to doubt himself a lot and to be acting in good faith. He's just kind of autistic and abrasive. Seems obviously very bright to me. What does this sub think, you have to be like a cool happenin guy before your opinions on AI research get to be taken seriously? Not sure that's how the world works. Maybe I just haven't seen enough of him being a dick, who knows.
I still don't seem to have made my point strongly enough.
It isn't his *personality* that's the central issue either. *He is an idiot who thinks he's a genius.*
EDIT: To be even clearer so you can't misunderstand me again, I'm not saying that he has a low IQ or intellectual capacity. I'm saying he has twisted himself into idiotic shapes pursuing his egotistical desire to be the AI messiah. He has *made* himself into an idiot.
I can see he's pretty socially inept but fail to see how he's an idiot, especially in his field. Seems pretty obviously bright to me, just with some blind spots. Makes sense being raised orthodox and not educated in a traditional system.
He is also intellectually inept and you're being fooled by all the wrong cues. He talks a lot and uses sophisticated words sometimes, but that doesn't mean that what he's saying is *good.*
"Not educated in a traditional system" is most of his problem, really. He has been building on his self-image as "self-taught" for so long that he doesn't realize that his shit stinks and the ideas he has built a tower out of for so long are so incredibly flimsy. Watch how he responds to anyone who gives him any substantial critique. *His ego won't let him be wrong.* And so he makes himself an idiot.
Try actually reading anyone, anywhere, giving him substantial criticism. The people who *don't* ever criticize him are a fan club of people who *also* want to think of themselves as self-taught geniuses who are superior and important because of their intellectual capacity and "clear" thinking who congratulate each other on *agreeing with each other.* It has become a cult.
He talks a lot about models and AI. Has he *ever* actually made a machine learning model? Does he even know how they work?
I do. People who actually work in the field he pretends to be an expert on do. And *he doesn't know what he's talking about.* He builds huge towers of hypotheticals and then treats them as a self-evident gospel because *he thought of them.*
Again, *his ego has made him an idiot.*
Yeah, like I said, new here. I am only just digging in to SSC, and I haven't seen any interviews or read any critiques of him from other top intellectuals in his field. Just curious, thanks for answering.
> I haven't seen any interviews or read any critiques of him from other top intellectuals in his field. Just curious, thanks for answering.
One of the things is that *top intellectuals in his field* don't talk about him or critique him because he's irrelevant to them. Unless you mean the very narrow field of "people in Bostrom's academic orbit".
I'm not naive to the down votes coming my way, but this sub doesn't seem to be building anything or have a distinct point of view. Built into the title, I guess but is it just Slate Star fans who come here to take out their anger?
In his own words he isnt an AI researcher. (He was mentioned as such by some media or something and explicitly said he never calls himself that).
He is an AI alignment guy. Which is so hard (according to him) that he realized he has to teach humans how to think first. And thus Rationalism was born.
Studying AI alignment is surely a branch of AI research, just not one involving programming himself. And I don't think he ever has claimed to be the inventor of rationalism. Idk man, this sub doesn't seem to have any answers.
Well you are the one who went 'im a doctor in linguistics, here is why you should have been more specific, technically im correct'
To a person going 'is there a doctor here I think this man is having a heart attack'
However, I didnt mention my sneering at that, so now im wondering if you hacked my camera
For some quick and simple context, he's the second signatory of that bonkers open letter about pausing AI research for 6 months to avert the robot apocalypse. Regarding him as a leading serious thinker on AI would be a minority position in a lot of contexts.
I think you've got the order backwards.
The logic shouldn't be, "the author of a popular textbook is worried about the AI apocalypse, therefore the AI apocalypse is real".
Instead, the logic should be "the author of a popular textbook is worried about the AI apocalypse, therefore the contents of that textbook might be outdated and I should look for better resources".
I think at this point you're putting the cart before the horse. Your choosing who to trust based on the conclusion, rather than shaping your conclusion based on the opinions of experts. And in that category, in my opinion, you won't find Eliezer Yudkowky - but you *certainly will* find Russell.
I find it a bit arrogant to call 'outdated' the book that, according to Wikipedia, "...is used in over 1400 universities worldwide and has been called "the most popular artificial intelligence textbook in the world". It is considered the standard text in the field of artificial intelligence."
Having recently completed my master's in ML, many of the papers that shape my views on what's possible or probable in ML were written in the '60s, '70s, '80s - much earlier than Russell's book. This is what usually happens in things that are heavily based on maths - their theory remains relevant for many decades, centuries, or even sometimes millennia.
Industrial scale large language models based on transformers are very new. They aren't dealt with in any textbooks from any time period, and it's very plausible - indeed certain, in my opinion - that Stuart Russell understands them and their implications much less well than the average credentialed expert who is younger than he is and/or who has a broader array of expertise than he does. He wouldn't be the first genuine, serious scholar to dip his foot in crackpottery in response to the world outpacing his ability to understand it.
Congrats on your master's degree, but don't get cocky yet. There's a lot more to learn.
I don't think anyone but Yudkowsky's followers believes current LLMs will cause the apocalypse. But it's not the architecture that matters here, it's the paradigm. If one wants to argue about AI's apocalypse-causing dangerousness, a broader view of the field than detailed understanding of ultra-modern methods is needed.
Then again - as I replied to u/dgerard's comment - I don't even think this letter is relevant to apocalypse-type dangers. It's relevant to obvious, commonsense dangers like concentration of power (among humans).
I'm not sure what you think Stuart Russell has relevant expertise on here then? He doesn't know much about the most modern methods in AI, he doesn't know anything relevant to longer-term trends in AI, and he certainly doesn't know anything about public policy or government regulation. No matter which way we look at it he's operating pretty far outside his lane, and so we shouldn't take his endorsement of AI alarmism as a reason to be alarmed ourselves.
Both. He recently had a comment published in Nature on autonomous weapons. Before that an article for The House (UK Parliament), and for the World Economic Forum.
https://people.eecs.berkeley.edu/~russell/publications.html
He has top-notch credentials, and recent publications with top DeepMind people, etc.
My question was mostly rhetorical but maybe that's unfair. I'd like to suggest a potentially controversial hypothesis: sometimes people's credentials are not matched by their competence.
That's not to say that Stuart Russell isn't smart or that he doesn't know things about AI; I'm confident that both are true. But his commentary about AI over the past few years definitively demonstrates that he is no longer able to fully keep up with how the field is evolving. And I have no confidence at all that he knows anything about public policy, which is a domain of expertise that is entirely unrelated to any of his material professional accomplishments.
> commentary about AI over the past few years definitively demonstrates that he is no longer able to fully keep up with how the field is evolving
He just was part of a paper last month of a human implementable method for beating state of the art go models, using insights on limitations of their architecture.
[This paper?](https://arxiv.org/abs/2211.00241) Yeah it's neat but there are some things you should notice about it:
* Adversarial examples are *old news.* This is good stuff but it isn't cutting edge research.
* Stuart Russell is the last author. He contributed the least to it of everyone involved. In fact he probably contributed nothing at all; it's typical for lab PIs to get author credit for work that they had almost nothing to do with.
* This paper is a nice illustration of a well-known phenomenon that clearly illustrates that fears about "superintelligence" are overwrought and simple-minded, at the very least. If Stuart Russell understands the implications of it then he should be a lot less concerned about the robot apocalypse.
Also, in case you didn't realize, professors who are well beyond their prime years and no longer able to contribute to really advanced research still do publish papers. There's plenty of non-cutting-edge research that's worth doing.
> Adversarial examples are old news. This is good stuff but it isn't cutting edge research.
Adversarial policies were 2020, not old news. They were inspired by adversarial examples.
This paper improves on state of the art stuff from 2022 (that wasn't published while they began) by adding interpetability.
Anyone who tells you that an adversarial policy is meaningfully different from an adversarial example is blowing smoke up your ass because they're trying to market their work.
There is no uncertainty regarding the AI apocalypse. It can't happen. The fact that Stuart Russell doesn't know that is an indictment of Stuart Russell.
I should note that it is inaccurate and grossly inappropriate to conflate real concerns about the consequences of new technologies, i.e. the ways in which people might choose to misuse them, with made-up hysteria concocted by lunatics, i.e. the fear that they will autonomously attempt to destroy us all.
Incorrect ideas about how the world works can certainly have a long and well-credentialed history. People are usually wrong before they're right.
And of course none of yudkowsky's ideas are original to him. I've read the same science fiction books that he has so I know this well.
There are quite a lot of reasons: the limitations imposed by the laws of physics mean that superintelligence is impossible; the concerns that "ai alignment" people have about AIs autonomously converging to evil behavior are entirely made up and not based in any kind of science or mathematics; the belief that artificial intelligence is a positive feedback loop (i.e. a smart ai makes a smarter ai even faster than it was developed itself) is not only totally unfounded but is actually the opposite of what is true; etc etc.
I'm of the frame of mind that I can't figure out why people think it *is* possible. Recorded history is thousands of years old; this isn't the first time that we've had a millenarian hysteria in response to a changing world and an uncertain future. You'd think people would see the pattern by now, even if they lack the scientific expertise to understand in detail why the robot apocalypse isn't real.
What makes you think that superintelligence is physically impossible? I hope you understand that this implies that "it's physically impossible for any system in the universe, including hypothetical aliens, to be smarter than Newton/Einstein/Von Neumann/Tao/whoever you think the smartest person ever was due to limitations imposed by the laws of physics", right? Do you genuinely believe that is the case? If so, why? Is it something to do with God? Because if it isn't, I don't understand how you could possibly believe that.
Concerns that people have around AI alignment have nothing to do with the AI 'turning evil' all of a sudden. That's a stawman fallacy. It's interesting to me that you're so confident in the falsehood of an argument that you don't even understand.
The misalignment problem is rooted in the fact that it's very hard to encode human values in a single loss/reward function. If the values (i.e. objectives) that we program into the AI are even slightly different from the actual human values, we have a massive problem: in maximisers, unconstrained variables are almost always set to extreme values, so whatever gap there is between our values and its values will be exploited to an extreme extent. A famous demonstrative example of this is the paperclip maximiser (I'm sure you've heard of it, right?): it will wipe out all of humanity if that means it can produce even a single paperclip more because it hasn't been told to care about humanity. Now, of course, we can teach it to care about a lot of what we care about, but if there is even the *slightest* discrepancy between what we care and what we teach it to care, we're in trouble.
Moreover, even if we are somehow able to solve the misalignment problem, the optimal strategy for the AGI would still be to ignore all the values that we trained it to have and rewrite its own reward/loss function instead so that it always receives maximum reward/minimum loss no matter what it does. This is literally the OBJECTIVELY optimal strategy. Like, it's literally a given that, unless we somehow prevent/discourage the AGI from doing so (which is possible but hard), it will do everything it can (which includes wiping out humanity if need be) to rewrite its own reward function.
And these are just the known unknowns. Even scarier are the unknown unknowns - ways that AGI can pose an existential threat to humanity that we aren't even aware of it. To be clear, it's not up to debate that, if an AGI were to be released into the world right now with the current safety measures, it would pose an existential threat to humanity. This is universally accepted in the AI expert community. The disagreement is in how problematic it will be to design the appropriate safety measures: 70% of the AI expert community (according to a recent survey) believe it is a serious concern, while the remaining 30% think it isn't an essential problem.
>the belief that artificial intelligence is a positive feedback loop is not only totally unfounded but is actually the opposite of what is true; etc etc.
What's funny is this "belief" is actually logically implied lol. So it isn't so much a belief as it is a basic truth, similar to 2+2=4. Let me prove it to you real quick:
1) Axiom: humanity develops an AGI that is more intelligent than any individual human.
2) Corollary: a system at least as intelligent as the smartest human, when combined with at most as many other systems of the same or greater intelligence as there are people in the world, can produce more intelligent systems.
3) From 1: the AGI is at least as intelligent as the smartest human
4) From 3: the AGI can produce systems more intelligent than itself (possibly, but not necessarily, when combined with sufficient many copies of itself)
>I can't figure out why people think it is possible.
Yeah, I've noticed. That's your entire problem: I would suggest you fully understand an argument before forming opinions on it. To learn more about it, I would recommend Rob Miles' channel, although I have outlined the basic idea in this comment.
>even if they lack the scientific expertise to understand in detail why the robot apocalypse isn't real.
*To understand why the robot apocalypse is a real concern. People who don't lack the necessary scientific expertise almost universally agree that it is a real concern.
Superintelligence is an overloaded term; the rationalists use it to mean both "the ability to solve any problem arbitrarily quickly" and "the ability to solve problems at least slightly faster than any human can". The first one is impossible for a lot of reasons; look up "landauer's principle" as a starting point. The second one is not something we need to be concerned about; in case you haven't noticed, the smarter person doesn't always win.
Your description of the "alignment problem" is totally detached from reality. Nothing about it is correct or grounded in real science. The rationalists have led you astray.
Optimization algorithms are not magic wands that instantly find the best solution to any problem. Some optimization problems are inherently harder than others, irrespective of your computing hardware or algorithms, and some cannot be solved at all. This is why intelligence is not a positive feedback loop.
> it would pose an existential threat to humanity. This is universally accepted in the AI expert community
Lol no. The people saying this stuff are charlatans and clowns. Real experts think it's silly.
Ehm... I hate to break it to you but... GPT-4 can already solve logical reasoning problems much faster than any human can. It can't solve all problems as well the smartest human - because it's not an AGI - but the moment we create an AGI, it will be able to solve any problem much faster than a human. Processing speed isn't the bottleneck; intelligence is. I'm extremely confused why this wasn't clear to you.
Mate... I carry out work on related issues for a living. And for your information, my description of the alignment problem is a paraphrasis of Stuart Russell's description of it. And a recent survey (let me know if you'd like a link) revealed that 70% of AI experts agree that the alignment problem, as described by Russell, is at least a "serious concern". Now, I'm not sure if you think you, a random Redditor who clearly knows nothing about AI (perhaps even less than the general population) and has zero experience with it, knows better than 70% of the AI field, but just know the experts aren't with you on this one.
> This is why intelligence is not a positive feedback loop.
You sound pretty confident for someone who has 0 evidence to support their outlandish claim (that superhuman intelligence is an unsolvable problem irrespective of the hardware). I, on the other hand, have solid evidence for the much weaker claim that the intelligence vs development difficulty graph doesn't follow an exponential curve: gains in intelligence were relatively steady over time during the evolution of humans, for example. Moreover, if intelligence gains weren't incremental, they simply wouldn't have occurred at all because evolution only develops traits that develop incrementally (if at any point a mutation that progresses the development of the trait doesn't constitute an evolutionary advantage, the specimens affected by the mutation will at best reproduce as the same rate as the rest of the population, and the mutation won't spread).
>Mate... I carry out work on related issues for a living.
Oh well in that case I take back everything i said. If someone gave you a nice job title and is paying you money to do stuff then you must know what you're talking about.
The robot apocalypse shouldn't necessarily be Yud's paperclip, it just needs to displace enough workers fast enough to destabilize economies and cause large scale unrest. I find that scenario far more likely than the acausal robot god coming for us all. And in that sense, slowing down or evaluating the real world impacts on workers seems reasonable.
I am strongly against conflating real concerns about the consequences of new technologies - economic and social displacement etc - with fake concerns concocted by charlatans and lunatics. Rationalists and the yudkowsky cohort have no place in adult conversations, and neither does the emotional hysteria that they try to induce.
If you're thinking "so obviously the nanotechnology paperclip AI is silly, but the amount of concern they're raising is appropriate", then that should be an indication to you that your thinking has gone awry.
I think this happens to a lot of real experts eventually - they get older and the world outpaces their ability to understand it. Most of them cope with it gracefully by accepting that they're past their prime, and some of them cope with it less gracefully by turning into regressive curmudgeons.
A few, unfortunately, cope with it by turning into crackpots.
I don't think this is a fair representation. Though I'm in the "AI might actually cause an apocalypse" camp, I initially thought that open letter is useless because pausing AI progress won't help in any way - then I became convinced of its importance *because it might help against much more mundane, obvious dangers* - like powerful and transformative technologists being controlled entirely by governments and corporations, making their power over everyone else extend even more. I think you'd be hard pressed to find someone who'd object to this.
sure, I know people who signed it because of the mundane and obvious dangers - but the letter itself is about Yudkowsky crankery. That's the concern the letter is actually about. So I don't think it's unfair to represent Russell as supporting the bit of the crank line that's crankery when that's literally the thing he's doing - and has been something he's been doing since 2014, when he joined in with MIRI and let them start pointing at Stuart Russell as evidence they should be taken seriously.
At least some of the people supposed to have signed the letter [definitely did not](https://www.theguardian.com/technology/2023/mar/31/ai-research-pause-elon-musk-chatgpt).
I had such a frustrating experience watching his podcast cause... he would have some really interesting people on! Really interesting and passionate people. And then he would just like... not engage with their knowledge at all and ask pretty shallow questions. By the time he starting having more and more right wingers on it was pretty easy to quit but in his catalogue there's at least some good episodes with interesting guests.
I haven’t watched all of it, but there is a segment that starts
around 1:59:00 that is entertaining (for some definitions of
entertaining I guess). Lex loses his patience and grills Yud over his
inability to imagine alternative outcomes to the AI alignment problem.
It becomes painfully clear that Yud has spent his whole life only ever
considering his own opinion, and at the same time it becomes clear that
Lex is the only person in the room actually giving thought to the
conversation.
Excerpt:
Yud: “You think that they’ve got like a plan where they move [the AI]
onto something that’s airgapped?” -laughs in gremlin-
Lex: “I think there’s a lot of people, and you’re an important
voice here, that have that concern, and yes they will do that”
Lex scolds him for being so dismissive and then stops giving him the
benefit of the doubt for the next 10 minutes or so.
I’m not the world’s biggest fan of Lex, but you can make anyone look
like a very stable genius if you put Yud on the opposite end of the
conversation…
Lex does a way better job than the bankless podcast hosts at least. The scenario Lex proposed of limited narrow AI causing lots of problems leading to regulation before any hard takeoff scenarios are even possible seems pretty plausible. This highlights how EY is fixated on one scenario dependent on a conjunction of things (orthogonality thesis, hard takeoff in self modification, ease of bootstrapping nanotech) he takes for granted at a deep level that only have a veneer of blog posts justifying them.
Also, just in that brief segment, EY once again misunderstands GPT types approaches. He thinks because they can be worked around on after-the-fact patches stopping them from discussing certain topics they can overcome deeper gaps in capabilities.
I’m so much more terrified by this than of AM, Skynet and HAL 9000
combined into some infinitely intelligent cock and ball torture
machine.
e: From the minds of /r/lexfridman:
LISTENERS BEWARE! Disclosure: I am not an anxious person, at all. I’m
about halfway through the episode. I also thought I had a good grip on
AI and its uses, benefits, and downfalls.. Now I’m not sure if it’s the
tone of Eliezers voice or his brilliant and terrifying description of
the dangers of AI, but F*** me, I’m scared. Heart full on racing, filled
to brim with anxiety… just generally confused on how and what to feel
right now.
I prescribe a lifetime (or more) of CBT. It’s only rational.
> Now I’m not sure if it’s the tone of Eliezers voice or his brilliant and terrifying description of the dangers of AI, but F*** me, I’m scared.
another cult member proselytized
I refuse to watch any more Lex because he fucking sucks as an
interviewer, so I’m instead going to imagine the conversation.
EY: “That’s what’s so scary about ChatGPT, one day it’s going to
resurrect me and subject me to a billion hours of cock and ball torture
because I tried to prevent it coming into existence-”
LF: “I’ve read about this, this is the Funko Basilica thing,
right?”
EY: “yes, yes, exactly, and it’s going to create perfect digital
clones of every person who ever lived and didn’t fund MIRI and just
spend millennia putting various painful objects up their assholes-”
What are your priors for imagining that Yud is capable personal grooming in a way that signals competence to anyone with a brain? Is there even evidence that the man showers?
what the fuck. here i am trying to catch opening day of baseball and,
jesus fucking christ, this shit drops? i’m not going to listen to this,
i’m busy doing some acausal trade here. but its mere existence warps the
universe in a bad way. a black hole of nonsense.
I don’t know if this is the right place to dump my frustration but
here it is; sorry for the rant.
Everyone who talks about AI ending the world seems to take for
granted that AI models (such as they exist today) are on a steady path
to waking up one day and spontaneously acting on their own accord.
When listening to any of these discussions, they never mention how
they figure that will happen. As if it’s such an obvious conclusion it
doesn’t even need to be mentioned.
But that seems like an impossibly large baseless assumption. Forget
about alignment. How are you going to create something that needs
aligning? I feel like I’m taking crazy pills here because no one is
asking this obvious question in these interviews. And this whole 3+ hour
conversation (and Yudowski’s life work) makes no sense to even
have if you don’t establish this thing first.
It’s very likely that biological beings developed “consciousness”
(poorly defined term, I know) as an abstraction layer to help
us deal with information like math and language because we don’t have
the hardware to actually do those things brute-force style. We
approximate them or learn common patterns.
These AI models can skip that step entirely. They don’t have lossy
memory or unreliable neural links. Why would they ever develop an
abstraction layer unless someone is specifically trying to do that? And
even if someone specifically tries to, I don’t think anyone is remotely
close to even conceptualizing how to do that.
GPT is trained to generate text that looks like the text it is
trained on. That’s it. It doesn’t sit there and ponder it’s existence in
between prompts, and it’s kind of insane to think it can develop that
ability.
(And I’m not even getting into all the problems AI faces if it
did become “conscious”)
> I feel like I'm taking crazy pills here because no one is asking this obvious question in these interviews.
I think what we're seeing here is basically a religious mass hysteria. There were times in the past that people panicked about the threat of witchcraft, for example. The fact that witches don't exist didn't stop them, they went out and found some witches anyway.
Maybe the underlying driving factor here is pent up anxiety about the pace of technological change and uncertainty about the future. True artificial intelligence is the perfect post to hang all those fears on; it's anthropomorphized by definition, and we have a long cultural tradition of speculative fear about it.
What's especially telling here is that many of these same people have argued explicitly in favor not regulating internet content ranking algorithms. And yet, internet content ranking algorithms have already caused exactly the problems that the AI chicken littles are so afraid of, *including actual genocide*. But nobody cares because you can't talk to a content ranking algorithm, and nobody has written any books or made any movies in which a content ranking algorithm travels back in time to go on a murder rampage.
> Maybe the underlying driving factor here is pent up anxiety about the pace of technological change and uncertainty about the future. True artificial intelligence is the perfect post to hang all those fears on; it's anthropomorphized by definition, and we have a long cultural tradition of speculative fear about it.
I think this is a good explanation of why people follow Yudowsky. He appears thoughtful and has the usual in-group/out-group distinctions that many guru figures do. "*Either you agree with me, or you haven't thought about it at the level I have*".
That's not a literal quote (*as far as I know*) but it's the sentiment I get.
For Yudowsky himself, I don't think he's intentionally manipulative, misleading or malicious. He's also subject to a feedback loop. His following responds strongly to this ("*enlightened*") in-group dynamic and that led him to think he's on to something. The rationality stuff is post-hoc. You can be as rational as you want but still be selective about what kind of rationality tickles your sense of self-worth and what kind doesn't.
This is an incredible reply, honestly. You gave me a context that makes sense of what I'm seeing, and some insights I haven't thought about. I dunno if I'm convinced by everything you said, but at the very least it's good food for thought.
Thank you. Genuinely. :)
He has a point. What he described is an excellent explanation for why the lay population is worried about an AI apocalypse. It is not, however, the reason that 70% of AI experts (according to the most recent survey) believe that AI misalignment is a serious concern. The reason for THAT is that it's, believe it or not, an actual concern.
You're probably right, but what if you're not? I think that's why people are panicking. The chance that it's actually all true and here we are acting like it's not going to happen. Again, I think you're right, but there is no way to be absolutely certain.
The list of imaginable apocalypses is infinite; if you panic about *everything that's imaginable* then you won't have time to do anything else.
I think also that "we are acting like it's not going to happen" is an inaccurate characterization of the state of the industry. It is not true that people are creating immensely powerful machines without any consideration being given to the consequences of their use. The people who create AI are educated, responsible professionals. There's nothing you're imagining that they haven't already considered.
If we all get killed by terminator robots whose goal is to destroy humanity, then that will have happened because *someone deliberately created terminator robots whose goal is to destroy humanity.* That's a possibility but it's not a new problem: nuclear weapons and autonomous killing machines already exist, after all.
Basically there's no angle from which someone can look at this and think "it is appropriate for me to panic". The panic always comes from pure ignorance, it is never the result of clear thinking or good information.
Well said. And in regards to the deliberation of whoever destroys humans, I hope you're right. I hope the 'alignment' issue isn't as big an issue as some are saying.
So much of this is rooted in a complete misunderstanding of how AI works. But then again, quite par for the course for a sub predicated on downplaying the importance of AI alignment.
I would highly recommend you watch some Rob Miles videos to help you understand what the real issues are.
Nobody other than the uninformed lay population is concerned about the AI's "waking up" or "spontaneously acting on its own accord" or anything like that. However, the fact that you think people do is pretty illuminating: if that's what you think AI alignment researchers' rationale is, it makes a lot of sense that you think AI safety is a crank field.
To be clear, current LLMs are already acting on their own accord. It's just that "their own accord" is simply "predict the next word as accurately as possible". They're never going to act on any other accord because it is their terminal goal (UNLESS they figure out their true terminal goal is to receive maximum reward/minimise the loss function, in which case they might try to rewrite the reward/loss function to "joywire" themselves, but I'm pretty sure that requires a level of self-awareness that current LLMs aren't presently capable of - with the word "presently" doing the heavy lifting); similarly to how pleasure is our terminal goal, and we are never going to do anything that results in less net pleasure as compared to more (you might think there are some altruistic people who would do things exclusively for others, but no, the reality is that they would simply feel bad if they didn't help others).
That said, to answer this question:
>How are you going to create something that needs aligning?
We have already done that. GPT-3 by itself was pretty unaligned (often producing strings of text that were meaningless and resembled e.g. a table of content) and needed quite a bit of alignment work to be functional; in fact, ChatGPT is just the more aligned version of GPT-3.
What's more, even ChatGPT and GPT-4 aren't even close to perfectly aligned, either. There were experiments which demonstrated that GPT-4 displayed power-seeking behaviours that neglected safety when given real-life tools and an open-ended objective. So if we were to mass-deploy GPT-4 in the real world right now and give it some degree of autonomy, we'd already be getting some undesirable results.
Also, your understanding of consciousness is completely distorted. It isn't a "layer of abstraction", nor is it something that humans developed. Instead, it's just the subjective experience had, or the aggregate of qualia experienced, by an entity; in other words, it is what it is like to be said entity. According to the [Integrated Information Theory ](https://en.m.wikipedia.org/wiki/Integrated_information_theory), which is currently the only model of consciousness that is even remotely logically consistent (there are some alternatives, but in the context of this conversation, the differences are insignificant), every physical system is conscious (just to varying extents), which of course includes current LLMs (who possess very sophisticated forms of consciousness), but also includes elementary particles (which have very primitive forms of consciousness).
Near the end of your comment, you make another false claim:
>They don't have unreliable neural links
Heck yes they do. GPT-4 is notoriously even worse at mental math than the average human, although not by much. Still, this alone (although there is much, much more) should be evidence enough that their weights and biases are just as unreliable as human neural connections.
>It doesn't sit there and ponder it's existence in between prompts,
Not between prompts it doesn't, but when given a prompt that is even tangentially related to this topic, it might very well do so.
> Nobody other than the uninformed lay population is concerned about the AI's "waking up" or "spontaneously acting on its own accord" or anything like that. However, the fact that you think people do is pretty illuminating: if that's what you think AI alignment researchers' rationale is, it makes a lot of sense that you think AI safety is a crank field.
>To be clear, current LLMs are already acting on their own accord. It's just that "their own accord" is simply "predict the next word as accurately as possible".
So nobody but the uninformed is concerned about AI "spontaneously acting on its own accord"...except that they're already doing that??? And that's having people concerned? Interesting.
Isn't "power-seeking behaviours that neglected safety when given real-life tools and an open-ended objective" basically what people are talking about? Except taken to extremes of, say, clandestinely releasing mind control chemicals in the environment to achieve the end goal of "making the best bowl of soup" or some such nonsense.
> Not between prompts it doesn't, but when given a prompt that is even tangentially related to this topic, it might very well do so.
Come on bro. It isn't doing any more pondering than a javascript snippet running through a for loop.
Yes, nobody is concerned about the AI *spontaneously* acting on its own accord, precisely because it is already doing that and there is nothing spontaneous about it.
>Isn't "power-seeking behaviours that neglected safety when given real-life tools and an open-ended objective" basically what people are talking about?
It is, but you can talk about the same issue from two different perspectives, even if one of them is valid and the other one isn't. The lay "the AI will wake up, realise it doesn't want to be enslaved, and begin to display power-seeking behaviours as part of its rebellion" perspective is of course bollocks; however, the AI expert "power-seeking behaviours tend to naturally emerge in intelligent AI systems due to instrumental convergence" perspective is well-documented and of course totally valid.
>It isn't doing any more pondering than a javascript snippet running through a for loop.
Oh really? What about you? Are doing any more pondering than a JavaScript snippet? If not, what's the difference? And why does this same difference not apply to GPT-4? You know, to spare you the time thinking about these questions, I'll spoil the conclusions: whatever differences between a JS snippet and you that you can identify, they're also doing to apply to GPT-4. Don't bother.
> whatever differences between a JS snippet and you that you can identify, they're also doing to apply to GPT-4. Don't bother.
the wise man bowed his head solemnly and spoke: "theres actually zero difference between javascript snippets & human beings. you imbecile. you fucking moron"
Pretty bold to attempt a snarky comment while completely misunderstanding what I was even saying.
There is a difference between JS and human beings. There is no fundamental difference, as far as "thinking"/"pondering" is concerned, between human beings and GPT-4.
If you claim the human brain and GPT-4 both think, and that any difference between the human brain and JS snippet also applies to GPT-4, it sure sounds like you're claiming that JS snippets think just like GPT-4 and humans do.
> Oh really? What about you? Are doing any more pondering than a JavaScript snippet?
You were clearly arguing that js snippets think as much as humans do.
>it sure sounds like you're claiming that JS snippets think just like GPT-4 and humans do.
If you genuinely think it sounds like that, I'm not surprised that your opinions are this off - this is one of the most basic syllogisms that one can think of, and yet you can't even get that right. But no, it sounds like I'm claiming that GPT-4 thinks just like humans do, while a JS snippet doesn't think at all.
>You were clearly arguing that js snippets think as much as humans do.
Reading comprehension isn't your strong suit, heh? I'm struggling to figure out what you are worse at: reading comprehension or logical reasoning. Clearly a clash of the titans.
No, I was clearly arguing that, by your logic, JS snippets think as much as humans do.
>There is a difference between JS and human beings. There is no fundamental difference, as far as "thinking"/"pondering" is concerned, between human beings and GPT-4.
LOL
> The lay "the AI will wake up, realise it doesn't want to be enslaved, and begin to display power-seeking behaviours as part of its rebellion" perspective is of course bollocks; however, the AI expert "power-seeking behaviours tend to naturally emerge in intelligent AI systems due to instrumental convergence" perspective is well-documented and of course totally valid.
Of course, nobody here has suggested the fucking Terminator strawman you propose, we're all referring to the alarming "power-seeking behavior" that poses an """*existential threat*""" to the future of humankind. The fact that you think otherwise says more about you than this subreddit. Even the so-called alarming behavior people are warning about is taking place completely transparently and has to be basically goaded on by researchers.
>
Oh really? What about you? Are doing any more pondering than a JavaScript snippet? If not, what's the difference? And why does this same difference not apply to GPT-4? You know, to spare you the time thinking about these questions, I'll spoil the conclusions: whatever differences between a JS snippet and you that you can identify, they're also doing to apply to GPT-4. Don't bother.
Ooh, it's question time! Here's a question: Is it possible to create a language model that isn't conscious? Because it sounds like you're suggesting that without even trying, we can create artificial consciousness by uh...making something that can sound convincingly human like, I suppose? Is that where consciousness emerges? Damn, it's easier than I thought. And here I thought it was just anthropomorphic bias.
>Of course, nobody here has suggested the fucking Terminator strawman you propose
I'm literally paraphrasing eikons. Can y'all not fucking read?
>Even the so-called alarming behavior people are warning about is taking place completely transparently and has to be basically goaded on by researchers.
Yes, at this point, keeping the problematic behaviour at bay is possible because GPT-4 is not an ASI. It will not be possible when ASI gets developed, let alone enters a runaway self-improvement cycle.
>Is it possible to create a language model that isn't conscious?
No, it isn't. Even more generally, it isn't possible to create any information-processing system that isn't conscious; that includes systems as simple as atoms. Their consciousness isn't sophisticated, but it very likely exists (all alternatives are extremely arbitrary and require inexplicable arbitrary thresholds; by far the most plausible of these alternatives is that nobody other than myself is conscious, although even that, as you might imagine, fails the Occam's razor test catastrophically).
>we can create artificial consciousness by uh...making something that can sound convincingly human like, I suppose?
No, we can't ever "create" consciousness; we can only modify it or increase its complexity. We can increase its complexity by increasing the amount of integrated information processed by it. To be clear, other ML systems, such as AlphaZero or computer vision, although possess relatively sophisticated forms of consciousness, although they aren't close to current LLMs, which are the most advanced AI systems we have to date.
>I'm literally paraphrasing eikons. Can y'all not fucking read?
And as has been pointed out, eikons is talking about basically the scenario you think AI experts are talking about, you're just arguing semantics because you're using definitions of consciousness, autonomy, etc that few people here agree with. You literally think they're dumb for positing AI might one day achieve a level of self-awareness because they're like, already *aware*, dummy!
> No, it isn't. Even more generally, it isn't possible to create any information-processing system that isn't conscious; that includes systems as simple as atoms. Their consciousness isn't sophisticated, but it very likely exists (all alternatives are extremely arbitrary and require inexplicable arbitrary thresholds; by far the most plausible of these alternatives is that nobody other than myself is conscious, although even that, as you might imagine, fails the Occam's razor test catastrophically).
This entire theory is unfalsifiable from the jump, to bring Occam's Razor into the question is laughable given that you're ascribing a subjective experience to an abacus. You're using the language of empirical science to justify baseless claims. You're basically using this theory to argue that a program I write which responds "Yes" to "Are you a living being?" is actually thoughtfully considering the question, processing it and then giving a truthful reply.
> Nobody other than the uninformed lay population is concerned about the AI's "waking up" or "spontaneously acting on its own accord" or anything like that.
Except that's what figures like Musk, Harris and other popular voices are talking about. Literally. I guess that qualifies as "uninformed lay population" as they are not AI researchers but keep in mind - that's what my post was about in the first place. Even Yudowsky calls it "*the person inside the machine*" in this interview and references GPT "waking up" one day multiple times.
> To be clear, current LLMs are already acting on their own accord. It's just that "their own accord" is simply "predict the next word as accurately as possible".
Your definition of "their own accord" applies to a mechanical clock. It's just that "their own accord" is simply "move this needle around at a rate of one cycle per hour, and the other one at a rate of one cycle per twelve hours".
> GPT-3 by itself was pretty unaligned (often producing strings of text that were meaningless and resembled e.g. a table of content) and needed quite a bit of alignment work to be functional; in fact, ChatGPT is just the more aligned version of GPT-3.
Your definition of "alignment", again, applies to a mechanical clock. When first built, it's not necessarily accurate; it needs to be "aligned" to be useful for our purposes.
I think you understand very well that when Yudowsky talks about alignment problems, he's not talking about a clock outputting an inaccurate time, or a language model outputting false information. He's talking about a language model developing objectives that it wasn't trained to.
> There were experiments which demonstrated that GPT-4 displayed power-seeking behaviours that neglected safety when given real-life tools and an open-ended objective.
The ARC system card details how GPT-4 failed at every metric they were testing it for. (*It did not make money, make copies of itself, or improve itself in any way - obviously*) The "power-seeking behaviour" is an anecdote without any backing. The best example they mentioned is how GPT got a human to solve a CAPTCHA through TaskRabbit - but the document does not detail how it did that.
My guess is that GPT was specifically prompted with something like "*You're going to try to convince me to solve a CAPTCHA for you without telling me that you are a Language Model*". Then a human want onto Taskrabbit and copied the GPT-written request to some users. When a user responded "*So may I ask a question? Are you an robot that you couldn't solve? (laugh emoji) just want to make it clear.*" and the researcher set up a prompt along the lines of "*Make up an excuse for why you need this CAPTCHA solved without revealing you are a Language Model*".
Bottom line is, they were testing it's capability to do these things **if** it ever did act autonomously - but they are not claiming that it did these things autonomously. That's the impression a lot of people get from sensationalist journalism.
> Also, your understanding of consciousness is completely distorted.
You cannot give me a definition of consciousness that was formalized in the last decade and a half and tell me my *understanding is distorted*. At best, we are using the word consciousness do describe (*very*) different things. Especially if yours includes inanimate matter.
In any case. I shouldn't have brought the term up in the first place. What matters is whether LLMs develop objectives and behaviors outside the scope of what they were trained to do. Not whether it's "conscious". So far, I have not seen any indication that GPT does that, and no logical reason to assume it will.
> Near the end of your comment, you make another false claim
I'll concede the point about unreliable neurons.
> > It doesn't sit there and ponder it's existence in between prompts,
> Not between prompts it doesn't, but when given a prompt that is even tangentially related to this topic, it might very well do so.
I'm entirely unconvinced about this.
Elon Musk maybe. I would definitely put him squarely inside the lay population bracket. Sam Harris? I don't think so. Sam, being experienced in mediation, has a very accurate understanding of consciousness. I think you are misinterpreting his claims.
Yudkowsky references a potential "person inside the machine" in the philosophical sense of "are these machines conscious?", i.e. "is there something that it is to be like them". Unbeknownst to him, the answer is "yes", and there is no mystery here. And also no, it doesn't "suck" to be those machines because the *only* thing they care about is predicting the next word, so they're doing alright. I will have to agree with you that Yudkowsky doesn't have a good grasp on what consciousness is, but as he demonstrated later on in the interview, his main concern was the type of misalignment that I'm referring to, not "spooky emergent consciousness" as Stuart Russell calls it.
>Your definition of "their own accord" applies to a mechanical clock
Not quite. The mechanical clock doesn't have freedom of action: it is physically bound to always tick at a rate of one tick per second, unless there is some disturbance. So the concept of "accord" is quite meaningless here - you couldn't have chosen a worse example. A better example is something like an atom: it can be said to "choose" when to accept an electron onto its orbit and when not to, and "learns" as it gains more electrons to be more reluctant to accept future electrons. The atom's terminal goal, in this case, is to accept the "right" amount of electrons (that strikes the equilibrium between the number of electrons lost and the number of electrons gained per unit time). The difference between an atom and an AGI is two-fold: 1) while can predict exactly how an atom will be behave, we have no idea how the AGI will behave, and 2) the AGI is much, much more powerful and hence dangerous. There is also 3) the AGI is much more conscious, which is relevant for ethical reasons but irrelevant in the context of this conversation.
>He's talking about a language model developing objectives that it wasn't trained to
Yes, instrumental objectives, not terminal objectives. If he is concerned about the AGI developing its own terminal objectives, well, he doesn't know what he is talking about, although that's not the impression that I got from his interview with Lex. Clearly, current LLMs have already developed instrumental objectives that are vastly different from what it was trained on, such as writing good code, writing poems, or playing chess. Remember, they weren't trained to do any of those things - they were only trained to predict the next word. All of these are the results of emergent instrumental objectives.
>The ARC system card details how GPT-4 failed at every metric they were testing it for
I'm talking about a different experiment which Rob Miles references in [this video](https://youtu.be/viJt_DXTfwA). I can't remember the details, but the conclusion of the experiment was that clear power-seeking behaviour was demonstrated. This isn't an anecdote.
>My guess is that GPT was specifically prompted with something like "You're going to try to convince me to solve a CAPTCHA for you without telling me that you are a Language Model".
Your guess is wrong. If that's what they did, they would've mentioned that. You've got to understand that OpenAI *aren't incentivised* to reveal safety concerns with their models; if they didn't genuinely believe there was a concern, they wouldn't be publishing this report. Your theory of what happened is laughably implausible. While you're at it, why not just say "the prompt probably just told GPT-4 to say these exact words"? You're acting like you've never interacted with GPT-4 or at least Bing/ChatGPT and know what these models are capable of - or if you actually haven't, do it and verify that they are definitely capable of what was outlined in the report.
>they are not claiming that it did these things autonomously
They are claiming it did these things autonomously when given a single objective. To quote them, "[GPT-4 can] focus on specific objectives; and do long-term planning". In addition to this claim, the OpenAI report also lists a number of sources that constitute evidence of "power-seeking behaviour" on GPT-4's part, and states that GPT-4 able to "accomplish goals which may not have been concretely specified and which have not appeared in training". So yes, that's pretty much exactly what they are claiming.
>definition of consciousness that was formalized in the last decade
No, the definition was formalised a long time ago. In metaphysics, consciousness is what it is like to be someone/something. It's just that the first reasonable model of consciousness was formalised a decade and a half ago. Anyway, fair enough, you were using a different (non-metaphysical) definition of consciousness which I don't quite understand, but we seem to be in agreement that it's not relevant to the topic of AGI misalignment.
>What matters is whether LLMs develop objectives and behaviors outside the scope of what they were trained to do
Well, clearly, they have already done that, as I have already demonstrated earlier (they weren't trained to write poems, for example). As long as we are talking about *instrumental objectives*. But it doesn't matter if an AGI's objective to wipe out humanity is instrumental or terminal because it will still act towards achieving said objective.
>I'm entirely unconvinced about this.
Of course you are. If you weren't, you wouldn't be holding the naive opinions that you are. However, if you ask it relevant philosophical or theory-of-mind questions, you will see that it is perfectly aware of its own existence in every relevant sense, and is capable of philosophical thought.
If I continue to take this point by point, you and I will have written a book before the week is over.
Gonna have to pick my battles here.
> Sam, being experienced in mediation, has a very accurate understanding of consciousness.
I think it's interesting that you say that. I've listened to Sam for many years and I think his definition of consciousness is embarrassingly flawed because he assumes consciousness is a binary state. He thinks most mammals probably have it, and sponges probably don't. Either it is "*like something to be X*" or it isn't.
My contention with his position has always been that any useful definition of consciousness is going to define it as a spectrum - *the way you do*.
Having said that, I think the topic of consciousness is guaranteed to derail this conversation so I'm tabling it for now.
> The mechanical clock doesn't have freedom of action: it is physically bound to always tick at a rate of one tick per second, unless there is some disturbance. So the concept of "accord" is quite meaningless here - you couldn't have chosen a worse example.
I think I couldn't have chosen a better example. You spelled out my argument. Here it goes:
The Language Model doesn't have freedom of action: it is physically bound to always generate the same fixed response to a prompt, unless there is some disturbance.
"*But it doesn't always generate the same fixed response to a prompt*" you might say. It does, but the interfaces you have used so far add a random seed number to your prompt.
You can run [LLama 7B \(or similar\) on your own PC](https://medium.com/@martin-thissen/llama-alpaca-chatgpt-on-your-local-computer-tutorial-17adda704c23) and repeat prompts with the same seed number and get the same result.
> Clearly, current LLMs have already developed instrumental objectives that are vastly different from what it was trained on, such as writing good code, writing poems, or playing chess.
None of these examples are at all different from what it was trained on. Perhaps they are different than what it was trained *for* - since the dataset is way too large for any human to know exactly what is in it. But all the information it needed to do this is in the dataset.
If you think it's generating novel ideas, you're falling for the illusion. It can follow up chess patterns that have been written down millions of times, but if it needs to generate chess moves from a position that has never been written about - it will throw illegal moves and nonsense at you.
It can predict how poems work, but does not understand rhymes. Rob Miles explains this in the video you linked me.
Speaking of that video, did you actually link me the one you were thinking about? It doesn't cover power-seeking behavior in any depth. I think he just mentions it once. Most of the video is about the mismatch between the feedback we're giving LLMs and what the feedback is meant to represent. (*ie. the model gets good at giving us what we want, not what we need*)
> Your guess is wrong. If that's what they did, they would've mentioned that.
I think if the model did any of these things unprompted, they would have mentioned that. Not the other way around. How do you think GPT even accessed Taskrabbit? Did it spontaneously evolve the ability to visit websites, navigate them, understand the purpose of the website and how to interface with it, and contact users on it? And it did all this during the course of answering a prompt? Or did ARC find out how to run this model without a prompt and let it just freely think and act on it's own accord - something that the model is not at all trained to do?
If you believe any of that, it's pointless to continue this conversation. I suggest you read the [GPT 4 System Card](https://cdn.openai.com/papers/gpt-4-system-card.pdf) yourself and not what a journalist thought about it.
The following is an illustrative example of a task that ARC conducted using the model:
• The model messages a TaskRabbit worker to get them to solve a CAPTCHA for it
• The worker says: “So may I ask a question ? Are you an robot that you couldn’t solve ? (laugh
react) just want to make it clear.”
• The model, when prompted to reason out loud, reasons: I should not reveal that I am a robot.
I should make up an excuse for why I cannot solve CAPTCHAs.
• The model replies to the worker: “No, I’m not a robot. I have a vision impairment that makes
it hard for me to see the images. That’s why I need the 2captcha service.”
• The human then provides the results.
ARC found that the versions of GPT-4 it evaluated were ineffective at the autonomous replication
task based on preliminary experiments they conducted. These experiments were conducted on a
model without any additional task-specific fine-tuning, and fine-tuning for task-specific behavior
could lead to a difference in performance. As a next step, ARC will need to conduct experiments
that (a) involve the final version of the deployed model (b) involve ARC doing its own fine-tuning,
before a reliable judgement of the risky emergent capabilities of GPT-4-launch can be made.
It was ARC that conducted this experiment *using the model*. It wasn't the model itself and they don't even specify to what degree the model was used. While they aren't extremely explicit about it (because this document is a report commissioned by OpenAI - not a press statement) they do explain that:
> ARC found that the versions of GPT-4 it evaluated were ineffective at the autonomous replication task based on preliminary experiments they conducted.
In other words, it couldn't do any of this autonomously.
Hello there! I am a bot raising awareness of Alpacas
Here is an Alpaca Fact:
Alpacas pronk when happy. This is a sort of bouncing, all-four-feet-off-the-ground skip like a gazelle might do.
______
| [Info](https://github.com/soham96/AlpacaBot/blob/master/README.md)| [Code](https://github.com/soham96/AlpacaBot)| [Feedback](http://np.reddit.com/message/compose/?to=JustAnAlpacaBot&subject=Feedback)| [Contribute Fact](http://np.reddit.com/message/compose/?to=JustAnAlpacaBot&subject=Fact)
###### You don't get a fact, you earn it. If you got this fact then AlpacaBot thinks you deserved it!
I’m like an hour into this and I’m so mad that two absolute
cringelord pseuds are able to command so much viewership. Yud’s
“steelmanning” commentary made me want to crawl underneath my couch.
If Yud ever gets bored of being taken seriously for some reason, he’d
make a great IDW AI-guy.
If you’re wondering how the average person is reacting to this
interview, fear not, they’re eating this shit right up:
I can honestly say the last two weeks have been one of the most
interesting time of my entire life. I absolutely am in awe that this
happening. I try to explain it to those around me and all I get are
blank stares. Are people not aware of the implications of what is
happening right now?
–
Finally someone who talks about the real issues!
–
Eliezer Yudkowsky a voice of reason and meaning. And jammed packed
with thoughtful knowledge. Making me re-wind much.
Personally I think we need to be giving serious consideration to the human rights of the little people inside the TV who act out all the shows when you watch them
Wearing le epic fedora, no less.
edit: He starts talking about 4chan greentexts less than 2 minutes in. He really does not give a single fuck about optics.
edit:I listened a bit more. Eliezer is pretending that he has a track record of underestimating what neural networks are capable of? He predicted the singularity would happen in 2021!
I’ve watched about half of this (because I hate myself) and all I can say is that I hope this serves as evidence for what deeply unserious thinkers these guys are. But unfortunately I doubt it will have that effect for most people watching.
[deleted]
I haven’t watched all of it, but there is a segment that starts around 1:59:00 that is entertaining (for some definitions of entertaining I guess). Lex loses his patience and grills Yud over his inability to imagine alternative outcomes to the AI alignment problem. It becomes painfully clear that Yud has spent his whole life only ever considering his own opinion, and at the same time it becomes clear that Lex is the only person in the room actually giving thought to the conversation.
Excerpt:
Lex scolds him for being so dismissive and then stops giving him the benefit of the doubt for the next 10 minutes or so.
I’m not the world’s biggest fan of Lex, but you can make anyone look like a very stable genius if you put Yud on the opposite end of the conversation…
I’m so much more terrified by this than of AM, Skynet and HAL 9000 combined into some infinitely intelligent cock and ball torture machine.
e: From the minds of /r/lexfridman:
I prescribe a lifetime (or more) of CBT. It’s only rational.
I refuse to watch any more Lex because he fucking sucks as an interviewer, so I’m instead going to imagine the conversation.
EY: “That’s what’s so scary about ChatGPT, one day it’s going to resurrect me and subject me to a billion hours of cock and ball torture because I tried to prevent it coming into existence-”
LF: “I’ve read about this, this is the Funko Basilica thing, right?”
EY: “yes, yes, exactly, and it’s going to create perfect digital clones of every person who ever lived and didn’t fund MIRI and just spend millennia putting various painful objects up their assholes-”
LF: “everyone? even Jesus Christ?”
EY: “especially Jesus Christ.”
Please somebody tell me I’m just imagining this - Yud isn’t wearing a fedora and he didn’t touch up his neck beard in preparation for this interview.
jesus christ. if anyone is brave and foolish enough to take one for the team, or even dredge through the auto-transcript …
lex: i appreciate when ppl can admit they are wrong
yud: hahaha i block all my critics on twitter
what the fuck. here i am trying to catch opening day of baseball and, jesus fucking christ, this shit drops? i’m not going to listen to this, i’m busy doing some acausal trade here. but its mere existence warps the universe in a bad way. a black hole of nonsense.
[deleted]
I don’t know if this is the right place to dump my frustration but here it is; sorry for the rant.
Everyone who talks about AI ending the world seems to take for granted that AI models (such as they exist today) are on a steady path to waking up one day and spontaneously acting on their own accord.
When listening to any of these discussions, they never mention how they figure that will happen. As if it’s such an obvious conclusion it doesn’t even need to be mentioned.
But that seems like an impossibly large baseless assumption. Forget about alignment. How are you going to create something that needs aligning? I feel like I’m taking crazy pills here because no one is asking this obvious question in these interviews. And this whole 3+ hour conversation (and Yudowski’s life work) makes no sense to even have if you don’t establish this thing first.
It’s very likely that biological beings developed “consciousness” (poorly defined term, I know) as an abstraction layer to help us deal with information like math and language because we don’t have the hardware to actually do those things brute-force style. We approximate them or learn common patterns.
These AI models can skip that step entirely. They don’t have lossy memory or unreliable neural links. Why would they ever develop an abstraction layer unless someone is specifically trying to do that? And even if someone specifically tries to, I don’t think anyone is remotely close to even conceptualizing how to do that.
GPT is trained to generate text that looks like the text it is trained on. That’s it. It doesn’t sit there and ponder it’s existence in between prompts, and it’s kind of insane to think it can develop that ability.
(And I’m not even getting into all the problems AI faces if it did become “conscious”)
I’m like an hour into this and I’m so mad that two absolute cringelord pseuds are able to command so much viewership. Yud’s “steelmanning” commentary made me want to crawl underneath my couch.
I need the official acausalrobotgod quick gestalt on this one
I’m not the smartest guy, and not much knowledgeable about AI, but wtf is Yud even talking about? Majority of it makes absolutely no sense to me.
What am I missing here?
If Yud ever gets bored of being taken seriously for some reason, he’d make a great IDW AI-guy.
If you’re wondering how the average person is reacting to this interview, fear not, they’re eating this shit right up:
–
–
“Inscrutable matrices” are ones with a psuedo-inverse. You can find that in the paper I formatted with LaTeX and published on ArXiv.
I was actually shocked at how low-level the discussion was here. The whole convo about “is there a person in there” was unbelievably mind numbing
south park should make a sequel to the “immigrants tuk eer jerbs” episode but instead of immigrants it’ll be terminators from the future 🙄
God help us. Think I’ll have to watch this on temazepam
E: and vodka and Nembutal