only computer touchers should be allowed to do phil
ignores remark but starts using technical abbreviations like AIXI,
NARS, AIKR.
I wonder if this was intentional, and how many people in lw missed
the moment of enlightenment after googling what those terms mean.
E: a scroll through the comments is nuts. No minds were changed, they
wished yud could have debated him, and they want to go after younger ai
researchers because their minds are more malleable to the miri ways.
Iirc there is research that shows debate doesnt actually change minds
but only makes people more entrenched in their positions, this seems
like a good example of it. (Which reminds me that imho the best way to
teach uni level subjects is teachers going ‘lets figure it out together’
and not ‘this is why you are wrong and im right’ but that is my personal
exp).
everyone says I am dumb for saying that I will suddenly become a god tomorrow! they'll soon see how wrong they were, as my iconoclasm is irrefutable proof that my insane conclusion is correct.
There will be no way to design a “safe AI”, just like there is no way
to require parents to only give birth to “safe baby” who will never
become a criminal.
You’re telling me Asimov’s fiction wasn’t true? This changes
everything
This comment is great for illustrating that they have adapted their
mentality to “how do we look less like a cult of
personality” rather than the “how do we be less like a
cult of personality”
The “friendly AI” approach advocated by Eliezer Yudkowsky has several
serious conceptual and theoretical problems, and is not accepted by most
AGI researchers. The AGI community has ignored it, not because it is
indisputable, but because people have not bothered to criticize it.
I came to this conclusion years ago and wrote off the agi safety
crowd:
“ If intelligence turns out to be adaptive (as believed by me and
many others), then a “friendly AI” will be mainly the result of proper
education, not proper design. There will be no way to design a “safe
AI”, just like there is no way to require parents to only give birth to
“safe baby” who will never become a criminal.”
Now that I have a kiddo, I am even more convinced that (a) miri etc
doesn’t even understand learning and intelligence and (b) you can’t
understand agi until you raise a kid yourself.
It’s such a rich experience, and this notion that we are going to
code agi instead of growing it is silly. Even current strong ai systems
are essentially grown, and the quality of the input is critical.
Which is no different than raising kids. You have to be careful what
the consume and intake, both food wise and info wise.
Don't you have *any* concerns that your kiddo will simulate your personality for the purpose of eternal torture?
If not, can I ask if they're of the age yet where you have to attend school plays or music recitals?
funnily enough, that quote actually describes the state of "adversarial defenses" in deep learning research pretty well. Basically, if you're worried about hackers fooling your neural network with carefully crafted inputs, there's only one way to defend your network against them -- to include such inputs in the training set, i.e. to "educate" the network about what it should do when it sees inputs that "look like" they're from a hacker.
Lots of people have tried to make architectural changes to defend against adversarial inputs, but generally people are able to find new attacks to get around the changes. The only really durable defenses (afaik) are (a) including adversarial inputs in the training set and (b) increasing the capacity of the network.
Our eyes are always moving a little bit. We aren't actually intaking pictures of the world, but a continuous stream of "video" if you will. That little eye movement all the time is preventing us from having the same trivial kinds of flaws. Our learning is much more robust.
This is an area that I rarely hear AI/ML people talk about. How the biology of us actually helps us have a more robust learning model. 21 million pictures in your AI model? oh that's cool, that's like what, the equivalent of 1 hour of a 3 year old's experience?
I think in the long term AI is interesting, it's just that the breathless excitement now is just so... dumb. And easy (and fun!) to shit on!
those are interesting hypotheses, do you have any data backing them up? I've seen adversarial examples crafted to survive all sorts of aggressive filters, including filters that blurred or shuffled the input image. heck, I've seen adversarial examples propagated through a [water simulation](https://arxiv.org/abs/1910.00935) (see Appendix E, section 8). I don't think visual saccades are necessarily the reason we don't experience them.
Everybody comes blustering into deep learning assuming they understand how deep neural networks work, and every time their un-tested assumptions turn out to be wrong.
Nope. I got nothing.
Humans aren’t vulnerable to those kinds of attacks - why is that? Why does static look like a “bird” to some networks?
I’m sure researchers have realized that the human eye intakes a ton of data, and that the training set is the weak point. It all makes sense but this isn’t the same as demonstrating actual results.
All I know is human brains are really different in versions key ways vs current deep learning. And I never hear people talk about that.
EDIT: also I have a kid. That’s a ton of empirical observation on how humans brains learn.
Yeah that's reasonable. And yeah, I totally agree that there isn't enough discussion of deep learning models and how they are and aren't like human thought.
I guess I'm kinda touchy about making sweeping generalizations about that stuff, because I wasted a lot of time chasing phantoms during my research. I prefer to try and be as concrete as possible now; we know that humans aren't affected by the same adversarial examples as DNNs, but we don't know why. Just because we live in our heads doesn't mean we actually understand them, you know? And the same for DNNs, just because we made them doesn't mean we understand them.
e: also, comparing the amount of images seen by humans vs deep networks could actually be an interesting line of research. I remember one person estimating that, in terms of *unique* images you experience during childhood, the number seen by a DNN might be about the same as those seen in a few months by a baby. but if you didn't care about uniqueness, then yeah I think a baby would have way more experience. But I'm not sure it makes sense to compare human experience and DNN training anyway...
*Disclaimer: I am not a "LWer", I'm a PhD student studying ML/AI and I am not defending LW-style safety arguments in this comment*
> If intelligence turns out to be adaptive (as believed by me and many others), then a “friendly AI” will be mainly the result of proper education, not proper design.
This was actually something I disagree with, and I think it's something that would now be seen as outdated in the AI research community.
Similar arguments were being made with regards to bias and fairness in machine learning systems, but nowadays it is largely accepted that you cannot write off this problem so easily. The resultant model from an ML pipeline is the product of the algorithm you use and the training data, but from a safety/fairness perspective the behaviour of the end system is of primary importance.
> you can’t understand agi until you raise a kid yourself.
Don't you think this is deeply anthropomorphising?
> this notion that we are going to code agi instead of growing it is silly.
No-one since the 80s/90s is suggesting that we are going to hand engineer any AI system, but this doesn't mean that we aren't going to design the way in which learning happens. For instance, in reinforcement learning we typically use a loss function based on "temporal difference" of expected reward. This means that we design the metric by which system is "graded". It then changes its internal processing mechanisms to maximise this metric. In supervised learning, we are often maximising classification accuracy. In unsupervised learning (for instance GPT-3), we use a metric designed to encourage the system to model the distribution of the data (we use probability theory based metrics that I won't explain here).
In all these approaches, we are optimising according to *an objective function* (i.e. the metric). This objective is not learned, and will never change (in the traditional AI paradigm). When you start to consider objective functions that can be learned things get complicated, and this is where the state of AI safety is nowadays. I recommend looking into *Cooperative Inverse Reinforcement Learning (Hadfield-Menell, et al.)* and *Reward Modelling (Leike et al.)*.
This is what has led Stuart Russell (one of the most prominent AI researchers in the world, and the author of the internationally used standard textbook on AI) to propose a paradigm shift in the way we conduct AI research. He thinks we shouldn't even be focusing on "intelligence", instead we should be focusing on making systems that are "beneficial" to some third-party (e.g. a human-in-the-loop approach). For a non-technical approach I recommend his book, *Human Compatible: AI and the Problem of Control*.
I'm not sure I quite understand your question - what is an intrinsic property? Do you mean; is a lack of fairness an inevitable consequence of the algorithms used?
It's worth noting that "fairness" is not a very well-defined concept. There are several formal definitions, but there are also impossibility results showing you cannot satisfy them all simultanously. Furthermore, these definitions are contested themselves.
Ultimately, I think the issue lies in the architecture of the system (and how the learning process is structured). Firstly, we do not provide adequate signal to the system for capturing our concept of "fairness" (in the sense that we send the system a lot of signal about, for example, a dog, when we build image classifiers). Secondly, most current systems have no motivation to "act fairly". In other words, there is no mechanism that pushes the system towards "fairer" solutions to a given problem.
“It is based on a highly speculative understanding about what kind of
‘AGI’ will be created.” Actually, it seems to me that my notion of AGI
is broader than yours.
This isn’t coherent. Why would his definition being “broader” make it
less speculative?
I think he's making a point something like this: if we're about to roll a dice and you predict it will be a 5, and I predict it will be less that 4, my prediction is broader, but yours is more speculative (in the sense that it has lower probability).
If his definition of AGI is "broader" in the sense that it applies to more types of system, then his definition is more likely to apply to the system that ends up being created.
Therefore, a broader definition is less speculative because it covers more possible outcomes.
The actual issue with broad definitions is they may contain too little information to be useful.
Yeah, but the LWers are pulling a sneaky one here. Their definition *is* broader in the sense that they speculate on more civilization-threatening scenarios, but then, because of their amazing "priors", they divert attention and resources into this *narrower* subset. If you got introduced to the concept of AGI through LessWrong, your understanding of AGI would actually be very narrow.
[removed]
I wonder if this was intentional, and how many people in lw missed the moment of enlightenment after googling what those terms mean.
E: a scroll through the comments is nuts. No minds were changed, they wished yud could have debated him, and they want to go after younger ai researchers because their minds are more malleable to the miri ways.
Iirc there is research that shows debate doesnt actually change minds but only makes people more entrenched in their positions, this seems like a good example of it. (Which reminds me that imho the best way to teach uni level subjects is teachers going ‘lets figure it out together’ and not ‘this is why you are wrong and im right’ but that is my personal exp).
[deleted]
You’re telling me Asimov’s fiction wasn’t true? This changes everything
This comment is great for illustrating that they have adapted their mentality to “how do we look less like a cult of personality” rather than the “how do we be less like a cult of personality”
https://www.lesswrong.com/posts/gJGjyWahWRyu9TEMC/muehlhauser-wang-dialogue?commentId=AwkZfxQrHJ76DLxW7
Great sneer
I came to this conclusion years ago and wrote off the agi safety crowd:
“ If intelligence turns out to be adaptive (as believed by me and many others), then a “friendly AI” will be mainly the result of proper education, not proper design. There will be no way to design a “safe AI”, just like there is no way to require parents to only give birth to “safe baby” who will never become a criminal.”
Now that I have a kiddo, I am even more convinced that (a) miri etc doesn’t even understand learning and intelligence and (b) you can’t understand agi until you raise a kid yourself.
It’s such a rich experience, and this notion that we are going to code agi instead of growing it is silly. Even current strong ai systems are essentially grown, and the quality of the input is critical.
Which is no different than raising kids. You have to be careful what the consume and intake, both food wise and info wise.
This isn’t coherent. Why would his definition being “broader” make it less speculative?