They also cite Paul Graham as an "extremely intelligent and dynamic thinker" in the prompt, so I think this may be a case of garbage in, garbage out...
I first became aware of the insidious AI plot to create paperclips during the physical act of love. A profound emptiness came over me. Luckily I was able to interpret these feelings correctly
I thought that the model must have been biased, even perhaps told explicitly to mention him, but I asked GPT-4 this prompt (omitting paulg and adding a request for 10 candidates):
>looking for a lesser known, but extremely intelligent and dynamic thinker, with lots of written material online, to summon via ChatGPT for a persona for a hypertutor -- thinking people like @samoburja or @mikeduncan or @MKBHD. Can you list 10 good candidates?
And it produced the following results:
1. Eliezer Yudkowsky (@ESYudkowsky) - Artificial intelligence, rationality, and effective altruism.
2. Venkatesh Rao (@vgr) - Strategy, technology, and the future of organizations.
3. Julia Galef (@juliagalef) - Rationality, decision-making, and cognitive biases.
4. Melanie Mitchell (@MelMitchell1) - Complexity science, artificial intelligence, and machine learning.
5. Nassim Nicholas Taleb (@nntaleb) - Probability, risk, and decision-making under uncertainty.
6. Zeynep Tufekci (@zeynep) - Technology, society, and the impact of social media.
7. Scott Alexander (@slatestarcodex) - Rationality, psychology, and various interdisciplinary topics.
8. Devon Zuegel (@devonzuegel) - Urban planning, economics, and community-building.
9. Tiago Forte (@fortelabs) - Personal productivity, knowledge management, and the future of work.
10. Gwern Branwen (@gwern) - Psychology, technology, and various interdisciplinary topics.
I'm surprised that so many of them are "rationalists" or AI researchers. Is it because there is a terminally online bias, some trigger word in the query (what the heck is a hypertutor -- or is it "lesser-known"?), or something about the training data that bends it that way?
Playing with it some more, replacing "hypertutor" with "tutor" and removing all suggestions produces a similar list. Removing "lesser known" produces suggestions such as Jordan Peterson and Elon Musk.
> Removing "lesser known" produces suggestions such as Jordan Peterson and Elon Musk.
Ugh, maybe part of the bias also comes from "extremely intelligent and dynamic thinker", it's not an entirely sane way to describe a person, even if they are highly intelligent, and it feeds into far-right (race realists) personality cult figures somehow?
Out of curiosity (Not really willing to pay for GPT-4 myself) could you try:
> looking for a mildly obscure, but well respected intellectual figure, who has written a significant amount of material freely available online, as candidate for ChatGPT to adopt as a tutor persona -- thinking people like @samoburja or @mikeduncan or @MKBHD. Can you list 10 good candidates?
EDIT: Nvm list looks similar in regular Chat GPT interestingly with my version of the prompt removing "@samoburja" produced less crazy results, but the results are still crazy with using the original prompt, but saying just "@mikeduncan or @MKBHD"; so it seems to be a confluence of a particularly bad request.
Based on the list of names, it seems the rest might have the problem of intellectual pseudo-supremacy that comes with being "more intelligent" than all the plebs of the Earth, though they're more "dynamic" than Jordan Peterson or Elon Musk just because it's for most public figures to get more infected with anti-woke brainworms than those two.
MKBHD seems to be a tech review youtuber which could be guiding it also. Mike Duncan is a popular historian (with mostly sensible opinions) so I don't think he'd be the name influencing it towards people like Yud and Rao.
In Google's C4 dataset:
readthesequences.com, 280k tokens, 0.0002% of all tokens
lesswrong.com, 1.1M tokens, 0.0007% of all tokens
hpmor.com, 120k tokens, 0.00008% of all tokens
On the other hand
rationalwiki.org, 3.1M tokens, 0.002% of all tokens
Additional evidence that AIs don’t understand what they’re saying.
It looks a bit like a “leopards ate my face” moment frankly, what type of sane person would want to summon a “hypertutor”?
MR. PRESIDENT
WE MUST NOT ALLOW
A 200,000 WORD BLOG POST GAP
A well-known but still somewhat less-known thinker. The “who the hell is this clown” of intellectuals.
I assume that’s because @paulg was in the prompt.
It would be funny if AI gets malignly out of control because it was trained on texts about AI threats.
Its hard to think that anyone would really find Yud an ideal tutor for literally anything.
Oh boy, has Yud been uploaded into ChatGPT?
Isn’t this evidence that AGIpocalypse is never gonna happen
ChatGPT just recommended someone I wouldn’t hire to clean toilets… Gosh, learning from a cult leader is a terrible idea!