Considering the utter failure of projects like SI/MIRI to produce any results outside Harry Potter fanfic, is any of the stuff they propose even feasible on a nuts and bolts AI level? Here’s Yud’s views on symbolic and connectionist AI for a start:
https://www.lesswrong.com/posts/juomoqiNzeAuq4JMm/logical-or-connectionist-ai
I don’t know- AI is a very fast-moving and ill-defined field, and I think even an expert who tries to predict where it’s going has a larger-than-usual chance of being completely incorrect.
That being said, Big Yud is so clearly bad at talking about AI that I trust him about as much to talk about AI as the average CS undergrad.
Examples from this piece:
We don’t yet even have a strong grounding in why neural networks work, afaik (feel free to correct if I’m off here, not fully up to date on the field). Like, lots of cutting-edge methods have no rationale beyond “lol it performs better on MNIST / CIFAR-10 / ImageNet.” And the complaint that knowledge bases are complicated? Has he ever seen training data? Data are, in general, awful.
If you believe that connectionist AI is a simple, reasonable thing, take a look at some of the examples of neural networks being tricked into making utterly bizarre image-misclassifications. They’re fantastic.
He’s talking about how progress in AI can be much slower than expected… in order to argue that we should be worried about immanent general AI? I mean, I know his point is that it would take a long time to develop “Friendliness theory,” but still, he’s undermining his own point.
(Doubly so because this is an essay about how general AI needs to come from a radically new idea we don’t even have yet, so what he proposes to do is develop a theory for an AI system we not only don’t have, but also don’t even understand the basis of? It’s like asking someone who doesn’t know what a stock is to help you write regulations for high-frequency trading.)
lol he’s complaining about people getting sucked into incorrect neural network hype but he’s bought into the “neural networks are totally biology” hype.
(Hot take: we should ban machine learning researchers from using the phrase “neural” until they can explain, at an undergraduate level, how a synapse works. They should also be banned from saying “artificial intelligence” until they have a consistent definition of what “intelligence” is, which, given that nobody does afaik, should take a while.)
I don’t know anything about SI/MIRI’s proposals or results. I’ve tended to follow Deep Learning papers for the past couple of years, and I haven’t recognized any work from them, if I’ve seen it. Doesn’t mean much, though… It’s a huge field.
I agree with the overall strategy he seems to be outlining in the OP link, of mixing logical/Bayesian systems with connectionist. There are interesting contemporary architectures which involve something like that: Use the perceptual and decision-making capabilities of neural nets to drive some kind of symbolic or probabilistic system. I think it’s the way forward.
You seem to be sceptical about AI-safety-research-by-amateurs rather than AI.