The fundamental issue with Big Yud’s position on this is that we have
issues that you could reasonably call “AI alignment”, and they are
solved (to the degree that they are solved) by people who aren’t him.
When Google’s image recognition system identified a black man as a
gorilla, they did not turn to MIRI for help. When OpenAI tried to get
ChatGPT not to use slurs in its responses, they did not call in Big Yud
as an outside consultant. It’s all well and good to think about
theoretical problems. But if you do that without showing any ability to
solve practical ones, it’s not really clear why the people working on
practical problems should even take the time to rebut your theoretical
claims.
> When Google's image recognition system identified a black man as a gorilla, they did not turn to MIRI for help. When OpenAI tried to get ChatGPT not to use slurs in its responses,
But those are fake woke problems. What happens when Facebook literally creates God?
>When Google's image recognition system identified a black man as a gorilla, they did not turn to MIRI for help. When OpenAI tried to get ChatGPT not to use slurs in its responses, they did not call in Big Yud as an outside consultant.
No, they just turned to Mechanical Turk instead :V
Sure. But that's sort of the point, isn't it? The companies don't particularly *want* to use Mechanical Turk, if you could give them a framework where they said "we want the AI to behave well in these respects" and then it just did that, they'd be overjoyed to save resources (even if it just meant using Mechanical Turk to label for something else). But Yud cannot provide anything like that. He doesn't think providing something like that is valuable, and is dismissive of people trying to solve that problem. He is off in his own private kingdom, and is mad that people outside that kingdom don't care about him.
As an aside, if you use 'error propagation' as an alias for 'algorithmic bias' you might get some confused looks from ML people, in an ML context it's very easy to assume you are talking about error backpropagation, i.e. how the information about the error measured at the output of the neural network gets propagated back throughout the network to facilitate supervised training, in a strictly technical sense.
Their use of Mechanical Turk is also directly in conflict with his views about AI alignment in general---Yudkowsky thinks alignment is a problem to be solved by clever game theory and real life is showing that alignment is a problem to be solved by truly massive amounts of data.
It's also telling that all of the complicated theory-building which pushes our understanding comes from stuff like neuroscience and interpretability.
As it turns out, if you want to understand how minds work... you have to actually put them under a microscope and at least try to study them empirically.
I missed this story somehow. So did they literally go to mechanical turk and say "hey we need to generate X,000,000 pieces of training data that aren't racist?"
[Basically.](https://time.com/6247678/openai-chatgpt-kenya-workers/) But that's about what you would expect, lol, the fact that technology can't substitute for human judgement is exactly the problem these guys are trying so desperately to solve.
but your real problems are thorny and controversial and boring and hard to fix. even worse, i do not stand to personally benefit, which is epistemically suboptimal! how about studying my arbitrary, but SCARY, extrapolations instead?
>When Google's image recognition system identified a black man as a gorilla, they did not turn to MIRI for help.
I mean calling these issues as AI misalignment would be kind of a stretch. These in my view are more human error: error pertaining to not understanding the biases enforced within the datasets. This isn't at all AI being misaligned, so long as misalignment describes a deviation from the intended goals, as much as it is a misunderstanding of the intended goals. This is [referenced](https://mindmatters.ai/2019/01/can-an-algorithm-be-racist/) by the Google engineer, who solved the issue, as well:
>The biggest challenges of AI often start when writing it makes us have to be very explicit about our goals, in a way that almost nothing else does — and sometimes, we don’t want to be that honest with ourselves..
And this is precisely why nobody even thinks about Yud when it comes to solving these issues because it is immediately obvious that the error was in the input biases. As such, correcting the biases, either via an inclusive dataset, or hardcoding the bias out are perfect and effective solutions.
> The biggest challenges of AI often start when writing it makes us have to be very explicit about our goals, in a way that almost nothing else does — and sometimes, we don’t want to be that honest with ourselves..
Isn't this ultimately the same problem as the paperclip maximizer though? The paper clip company *obviously* didn't want anyone harmed, they just forgot to specify that in their success criteria. Google *obviously* didn't want a racist image tagger, they just forgot to include that in their success criteria.
This is where the whole scary AGI stuff starts to break down. Getting explicit about what you want from a system is *always* a hard part of building the system. And broadly speaking, even determining what our overall values as a society are collectively is difficult -- the best tool we have is democracy, and people get frustrated with it because it is a slow, tedious, messy, process based tool that *doesn't have quick tricks.* (Well, there are a few -- e.g., ban gerrymandering -- but they're mostly iterative improvements rather than genie in a bottle magic.)
The paper clip metaphor is the same as the Google image tagger is the same as how to get corporations to balance their capitalist need to serve customers with their democracy need to not make the world a worse place in the process. The best we can do -- in all cases -- is set up a process for everyone to have input and then a process that creates and enforces hard rules based on something between a bare majority and a total consensus.
The rationalists and AGI nerds hate the answer because they mostly don't believe in democracy, being technocratic totalitarians / aristocrats at heart. The need to solve "AI alignment," besides being inspired by popular scifi movies from their childhood, is mostly about their need to rule the world because the rest of us plebs can't be trusted to make decisions.
>Isn't this ultimately the same problem as the paperclip maximizer though? The paper clip company obviously didn't want anyone harmed, they just forgot to specify that in their success criteria. Google obviously didn't want a racist image tagger, they just forgot to include that in their success criteria.
But, as with all thought experiments, the issue is overblown in the context of paperclip maximiser. It is, indeed, a very big problem if companies started putting complete trust in their algorithms and give them complete power to do whatever is necessary to attain their goals. In this case, as with paperclip maximiser, even programs with meaningless goals would result in utter destruction.
But this is why AIs are treated more as *intelligence* rather than enacters/enforcers for a lack of better word. In any ML model, the AI part is distinct from the *action* part. There is one part of the code that learns, which would be AI, and another part which executes the required action, which is just normal programming. So, in all these thought experiments, the issue isn't with AI being misaligned; but rather the programmer not being cautious with the range of executable actions. In particular, a ML model *learning* that the best way to maximise paperclips is to wipe out all humans, that's one thing; it's another thing for programmers to create a code in which a possible *action* is to wipe out humans. That's a very different problem which has nothing to do with AI.
Similarly, a ML model learning to be racist due to biased dataset isn't an issue, but moreso that the programmers didn't write enough tests to ensure that the executable search was really in par with the guidelines. The latter isn't a problem with AI, that is just bad programming. Now, yes, it is true that it is quite impossible for programmers to rule out all possible "bugs" and "errors", but that isn't unique to AI; that is true for all programming.
The quote you pulled -- it's even more clear in context -- has nothing to do with "bugs" or "errors" or "program actions." It's directly about human goals and honesty and translating them into goals.
This isn't about programmers forgetting to write some unit tests, this is about programmers -- and more importantly the entire company and society that enable them to perform this function -- simply *not considering (or not caring) what it means to not be racist.*
> error pertaining to not understanding the biases enforced within the datasets.
But the bias isn't "in the dataset" in some sort of inherent way. If you or I (or some hypothetical person who was perfectly naive) looked at the data, we would not make this mistake. The bias is in the interaction between the ML system and the dataset. And, yes, that means we can and do fix the problem by giving the machine a better dataset. But if Yud is right, and getting AI to do what we want is a matter of thinking of very clever algorithms, he really ought to have some notion of how to get *this* AI to do what we want by virtue of applying a clever algorithm.
>>The biggest challenges of AI often start when writing it makes us have to be very explicit about our goals, in a way that almost nothing else does — and sometimes, we don’t want to be that honest with ourselves..
Honestly, this seems like the *exact* sort of language Yud would use to explain why he thinks AI alignment is a problem. The notion of the paperclip maximizer is precisely that there is a difference between the goal we intended for the system to have (make some paperclips to use around the office) and the goal the system understood (make as many paperclips as possible).
>But the bias isn't "in the dataset" in some sort of inherent way.
The bias is very much "inherent" to the dataset. Consider the article I linked: They speak about how people usually googling about "three black teenagers" in the context of crimes created an inherent bias in the dataset where it correlated "three black teenagers" with criminals and ended up giving mugshots on search result. I feel like a very naive person would also end up making these correlations if they were presented with no understanding of the world besides the dataset.
(Now whether this is *really* inherent or not is merely irrelevant semantics I believe; for if it weren't *really* inherent, it wouldn't have changed my point.)
>But if Yud is right, and getting AI to do what we want is a matter of thinking of very clever algorithms, he really ought to have some notion of how to get this AI to do what we want by virtue of applying a clever algorithm.
I don't think we should be granting him the benefit of doubt regardless of if he has presented any alternative or not. He has to first present an example in which the supposed misalignment has occurred which isn't reducible to simple human-encoded biases or errors. Otherwise, he can just keep inflating his own importance by calling out imaginary problems.
>Honestly, this seems like the exact sort of language Yud would use to explain why he thinks AI alignment is a problem.
Oh I agree it is a problem much like heat death of the universe is a problem. My contention is that we have neither seen nor we would see a similar problem in the foreseeable future that isn't merely just programmers being ignorant about the datasets. Now if we are going to call *that* AI misalignment then we might as well call accidentally creating a virus AI misalignment as well (for here as well we have a mismatch between intended and executed goals).
> He has to first present an example in which the supposed misalignment has occurred which isn't reducible to simple human-encoded biases or errors.
I don't see how this follows at all. Consider a problem completely unrelated to AI: car crashes. We can, I think, say that absolutely no car crashes are *caused* by not having seatbelts in cars. They're caused by distracted drivers, or bad road conditions, or brake failures, or any number of other issues.
But seatbelts are nevertheless *enormously* effective at reducing the number of fatalities from car crashes. So we use seatbelts, even though their mechanism of operation is completely unrelated to the root causes of the issues they remedy. Similarly, if Yud could demonstrate a technique that made AI produce less biased outcomes, I wouldn't care if he thought AI bias happened because AIs were cursed by ghosts.
> My contention is that we have neither seen nor we would see a similar problem in the foreseeable future that isn't merely just programmers being ignorant about the datasets.
The entire *point* of modern AI techniques is that they produce results that programmers cannot know from simply analyzing the datasets. The reason we use AI for image recognition is that our own attempts to figure out explicit rules for "this image is a picture of a grape" and "this image is a picture of a tortoise" and "this image is a picture of a mountain" *do not work very well*, and "hand the computer a bunch of pre-processed stuff and have it figure it out" is the best we've come up with.
So, yes, the result we don't like where AI decides that "three black teenagers" should mean "pictures of mugshots" is a result of programmers being ignorant about the dataset. But so is the result we do like where it can figure out that I mean "Avatar" when I ask "what was that movie with the blue people that had the actress from the movie with the racoon" (ChatGPT, interestingly, answers that correctly immediately, while Google search thinks I'm talking about "Over the Hedge" for some reason). The whole thing AI does is explore datasets to find correlations we don't see. So, yes, AI researchers would absolutely love to have some way of telling it "here are facts about the world, don't give answers that contradict these facts" reliably. If Yud could actually deliver that, they'd be super happy to talk to him, because that would mean they could worry less about providing good training data, which is the major limitation on modern AI systems. The issue is that he can't do that, not that an approach that tries to do that is fundamentally flawed.
>I don't see how this follows at all. Consider a problem completely unrelated to AI: car crashes.
Oh, I think there is a miscommunication because I believe we are in agreement here. To translate to your analogy, if someone tells me that there is a certain element in my car which increases the risk of crash, I have two responses: I can either ask for an alternative/solution, or ask him to justify this claim. Similarly, with Yud, my suggestion is that he has to first establish that there are, indeed, some misalignment happening in the first place which are, exclusively, due to AI, and not some bad programming practices.
Surely, you are correct that asking for alternative is another counter: but this sort of counter already enables his position and we both know he isn't going to code or create anything. The dude is going to peddle the assumed-for-the-sake-of-debate assumption as accepted alarming premises which nobody is allowed to speak against.
>If Yud could actually deliver that, they'd be super happy to talk to him, because that would mean they could worry less about providing good training data, which is the major limitation on modern AI systems.
But that is something I am not willing to give to Yud easily. You are completely right: it is a very rational way to go about this. If there is a possible problem, even in far future, and there is someone who claims there is a remedy, we should be all ears.
But.. but.., that isn't what happens with Yud and many more pseudoexperts. A lot of people, like Yud, Deepak Chopra, or even Terrence Howard, they point out problems merely so that experts would dismissively agree to their overblown caricatures, and then they can mischaracterise those dismissive agreements as unspoken dangerous truths which experts are either ignorant or too scared to approach. In fact, I would argue that they very much want experts to defer to them for alternatives or solutions to such hyperbolic thought experiments, because one, it makes them engage with you, and second, they can peddle it as, "Experts are coming to me to solve this problem they are too dumb to fix".
> exclusively, due to AI, and not some bad programming practices.
I think that understates the disconnect between AI/ML and prior programming practices. The notion of an error that is caused by a flawed training dataset is a fairly novel one. This is not something that happens with compilers or HTML engines or email clients. It's something that happens because we now have a category of program that operates by taking one program, running it over a giant pile of data, and producing a new program that we could not have produced on our own.
You can be perfectly diligent in constructing that first program and still get a bad second program. Indeed, you can generate a new version of that second program that does not have whatever problem you're concerned about without changing the first program.
So in my view, these problems aren't a result of bad *programming* practices, they're a result of bad *data science* practices. And, yes, we've largely solved them with data science solutions. What I would say in defense of Yud is that I think we actually *don't* have good solutions to these things in terms of programming practices, and that it's not impossible that there are programming practices that could fix them.
> but this sort of counter already enables his position and we both know he isn't going to code or create anything.
Then I am happy to ignore him. My position on this is that of the engineer. I am a computer scientist, though not a AI/ML person by specialty. I acknowledge the theory the AI/ML people are doing as useful because it can create things (like generative AI tools) that we cannot create without it. I reject the theory MIRI and Yud are peddling precisely because it does not create things that we cannot create without it.
> If there is a possible problem, even in far future, and there is someone who claims there is a remedy, we should be all ears.
I don't think that's quite what I would say. It's more that, if there's someone who claims there is a potential problem, and that they've identified a theory that resolves that potential problem, we should listen to them *if* they can demonstrate novel and useful applications of their theory. If, for instance, Deepak Chopra could demonstrate, in a way that held up to empirical scrutiny, that his brand of woo cured cancer of the gallbladder, I'd be willing to listen to him explain the principles behind his brand of woo and try to integrate it into my way of thinking. But if he can't proactively do that, it's not even worth taking the time to refute him.
Basically, my stance on Yud is one giant "put up or shut up". If he can show me a technical artifact as impressive as ChatGPT or StableDiffusion based on his theory, I will acknowledge it as a useful theory. If he can't, I'm not even going to bother learning what his theory says, let alone deciding whether it's right, wrong, or even coherent.
Oh, so it’s a problem that people expressing their opinions aren’t
famous academic AI scientists? Not sure why being famous matters, but
it’s interesting that he of all people has an issue here.
the full post is much longer. he goes onto claim that any serious academics that do agree with him have strong incentives not to be public with their thoughts. so he's accusing them of stupidity or being cowardly! Because of course no smart, intellectually honest person could disagree with the Mighty Yud!
To be honest the longer his whole x-risk nonsense continues the closer he gets to straight-up becoming a conspiracy theorist and getting folded cleanly in with the Q loonies. We've already hit "the end is nigh and the world is not virtuous enough to save itself," and it increasingly feels like only a matter of time before he goes off with his most loyal followers to a compound in French Guyana or something. Albeit one with a lot of catering because clearly they're too important to form a proper commune.
No, I'm just trying to draw the thin line between "vaguely connected to the cultic milieu" and "actively shouts on street corners about the Illuminati blood rituals corrupting our children towards gender communism"
Yes no “serious research scientists” disagree with Yudowski at all…. mostly because they don’t take him seriously. But a few I found through some quick googling:
Francois Chollet, research scientist, inventor of the Keras deep learning library and primary contributor to Tensorflow.
Dr. Emily Bender, PHD in Computational Linguistics and VP of the Computational Linguistics Society,
Dr. Gary Marcus, profsssor of Neuroscience at NYU.
Dr. Margaret Mitchell, PHD Computer Scientist, professor of complexity theory.
Most of this nonsense seems to be driven by people in the EA/long term isn’t cult, from California.
Drinking game if you want to get wasted, search “ai alignment” on twitter and have a drink for every profile posting about how AI are going to kill us all that talks about crypto investing in their bio.
> Drinking game if you want to get wasted, search “ai alignment” on twitter and have a drink for every profile posting about how AI are going to kill us all that talks about crypto investing in their bio.
How to go from zero to having your stomach pumped in 30 seconds or less
that fucking guy is just like those quack inventors who made a “free
energy” machine that will power the entire event (or whatever): they are
sending a message telling everybody they are on their way and will
arrive anytime now… and people will see, oh they will see…
Yud would be well-served to provide *any demonstration at all* that his techniques are of any real value before complaining about people not engaging with him. Even if you grant he is entirely right about the need to align strong AI, why would you think there is any value in taking his opinion on how to do that when he cannot show any ability to align extremely weak AI, a task that by his own analysis should be *much easier*. It's like listening to some guy rambling about how quantum entanglement means souls are real when he can't give you a coherent account of how the double-slit experiment works or what it proves.
This is like a stone throw away from those people who say “i believe
vaccines cause autism but all reputable journals, scientific studies,
relevant academics or empirical evidence i found say otherwise. Can
somebody please link any bona fide sources to back up my claim?!?”
Yud would be a villain character who decides to prove all his
naysayers wrong by designing an AI that could wipe out humanity. Too bad
he doesn’t have the brains to do it.
I hesitate to dismiss someone working on an important problem, even
if they haven’t shown significant strides in solving that problem in
real world terms. Frankly, the rest of the field’s efforts in this
endeavor aren’t any better, not that that’s at all comforting. Most of
the AI field, if they care about this issue at all, seems to take the
position that if and when we accidentally develop an unfriendly AI,
we’ll just figure out some way to deal with it at that time. The problem
with that idea is that once we develop an AI sophisticated enough to be
a meaningful threat on some level, we will already be in a no man’s
land. Any talk of scenarios involving an unfriendly AGI seem worryingly
vague on the specifics. And that’s only those who acknowledge the
possibility of such a problem.
I suppose I should have said potentially important problem. Quite aside from worries about extinction level events, I think we can agree that the influence of AI in our society is only going to grow. Focusing on ensuring that they’re well aligned to our goals seems important, and it also seems like something we’re currently pretty bad at.
Important or potentially important problem ? Sure, in 200 years or more it will maybe become a potential problem. The current position of all the "experts" (as Yud would add the quotes) however is that it's science fiction.
The main thing always comes down to "once we develop AGI ...". It's like "once we develop faster than light and affordable space travel, we will need space regulations ! Better work on it NOW it's urgent!!!" Or "once we develop cold fusion it's gonna put so much power in the hands of terrorists, we should work on eradicating countries hosting terrorist organizations ASAP just in case they develop cold fusion !!!"
Like I get the potential issues. But you need to give more than an ultra hypothetical to start it off.
The fundamental issue with Big Yud’s position on this is that we have issues that you could reasonably call “AI alignment”, and they are solved (to the degree that they are solved) by people who aren’t him. When Google’s image recognition system identified a black man as a gorilla, they did not turn to MIRI for help. When OpenAI tried to get ChatGPT not to use slurs in its responses, they did not call in Big Yud as an outside consultant. It’s all well and good to think about theoretical problems. But if you do that without showing any ability to solve practical ones, it’s not really clear why the people working on practical problems should even take the time to rebut your theoretical claims.
I’m sorry…he’s arguing that he can’t publicly debate with AI scientists because they will lose too badly? Am I getting that right?
Oh, so it’s a problem that people expressing their opinions aren’t famous academic AI scientists? Not sure why being famous matters, but it’s interesting that he of all people has an issue here.
that fucking guy is just like those quack inventors who made a “free energy” machine that will power the entire event (or whatever): they are sending a message telling everybody they are on their way and will arrive anytime now… and people will see, oh they will see…
I’m glad he admits that actual AI thinkers don’t take him seriously.
This is like a stone throw away from those people who say “i believe vaccines cause autism but all reputable journals, scientific studies, relevant academics or empirical evidence i found say otherwise. Can somebody please link any bona fide sources to back up my claim?!?”
legend in his own mind, and deeply ignorant
Yud would be a villain character who decides to prove all his naysayers wrong by designing an AI that could wipe out humanity. Too bad he doesn’t have the brains to do it.
there is too much danger in AI. please remove three.
p.s. i am not a crackpot.
I hesitate to dismiss someone working on an important problem, even if they haven’t shown significant strides in solving that problem in real world terms. Frankly, the rest of the field’s efforts in this endeavor aren’t any better, not that that’s at all comforting. Most of the AI field, if they care about this issue at all, seems to take the position that if and when we accidentally develop an unfriendly AI, we’ll just figure out some way to deal with it at that time. The problem with that idea is that once we develop an AI sophisticated enough to be a meaningful threat on some level, we will already be in a no man’s land. Any talk of scenarios involving an unfriendly AGI seem worryingly vague on the specifics. And that’s only those who acknowledge the possibility of such a problem.
Nano bots are coming.