Weirdly for a man who is near to a sneer himself, Stephen “cellular automata” Wolfram had a good article about what LLMs do that scales in gnarl with the length of the article.

https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/

Francois Chollet (creator of Keras, big contributor to TensorFlow, Google researcher) always has great anti-hype takes. He’s about as balanced as you can get.

Other stuff/people to check out would be Gary Marcus (who is now a meme among AI boosters), Timnit Gebru, Emily Bender, Margaret Mitchell, Melanie Mitchell (no relation) and Arvind Narayanan. All AI researchers (Emily Bender is a linguist specialising in NLP) and all have great critical writings/papers on these subjects. I think sometimes they can be a little dismissive of deep learning, but not dismissive in the sense that they think these systems won’t do damage. They are all in agreement that it will be very bad, just for reasons different to Yud.

I’d also check out writings by Dan McQuillan, Dwayne Monroe and L.M. Sacasas (especially his stuff about the Borg Complex). Noah Giansiracusa has some good posts. The Tech Won’t Save Us by Paris Marx podcast has great episodes on AI (bringing on people like Gebru and McQuillan), but also just really great anti-tech/VC discussion in general.

You’d also do well to read some older media theory. Marshall McLuhan and Neil Postman might be good starts.

Yann LeCun (head of Meta AI, i.e. bit of a fucking red flag) is relatively tethered to reality, and he does definitely know what he’s talking about, but I also really dislike him because he’s incredibly dismissive, insisting the things he builds are incapable of causing harm. Extremely corporate.

I might periodically edit this with other names as they come to me. Keep in mind though that this is not a reading list to counter Yudkowsky-esque doomerism. That requires a different set of texts, which I’m not sure are necessarily on topic for this post. The stuff above is more for people looking to drown out the clamour and determinism specifically from VCs.

Besides reading, you should also just play around with LLMs like ChatGPT or Bard, and try to probe their limits. One way you can do this is take a problem that’s shown up in their training set thousands of time, tweak it so it is obviously different or trivial in some way, and then ask the LLM for the solution. For example:

Write a Python function that, given another function f(x) and input i, returns True if f(i) terminates and runs forever otherwise.

GPT-4 spits out the standard explanation of the halting problem at this. It’s fairly easy to come up with other examples that trip it up.

[deleted]

Since I’m annoyed by people generally being ignorant about the actual fundamental limitations of LLMs, I might as well shill Shaking the foundations: delusions in sequence models for interaction and control. It’s not a highly technical paper by the standards of AI research, but does require at least some vague intuition about statistics and comfort with mathematical notation to get through (I can say that, because that’s what I have).

It’s a paper about a fairly specific detail rather than an overview, so it won’t leave you feeling less lost as such, and even if you understand it well, it’ll take a bit of extra effort to get from it to specifically thinking about LLMs producing text pretending to be a human.

But my sales pitch is, maybe the subject will get you curious because it’s about something essential that we know is missing from our current LLMs and will remain missing no matter how much we scale them; and it’s a mathematically rigorous treatment, so even if it takes a while to build up the background to handle it, that’s exactly the background that you need to think clearly about statistical models.

The Computerphile guys on YT have some pretty good roundups and summaries on how LLMs work at a very low level.

I think one of them is definitely a LW Yudkowskian kinda guy, he always talks about ‘alignment problems’ and is obsessed with AI Safety, but if you catch one of the ones where he breaks down exactly what the model is doing it’s not too annoying, because he explains things well.

You’d probably want something a bit meatier if you’ve got a baseline understanding, but I think this video from Solar Sands is a fairly decent, broad strokes introduction to AI and related issues that isn’t coming from either of the perspectives you mention. It has more of a technical focus on the likes of Midjourney and Stable Diffusion than Chat-GPT, however.

If you haven’t read it, this old Norvig essay is absolutely essential reading, regardless of how you feel about him now. It’s from 2010 or 2011, so at this point statistical techniques were in wide use, but we hadn’t had the major breakthroughs of word2vec or transformers yet, which I think makes some of his predictions a lot more salient.

https://norvig.com/chomsky.html

Dan McQuillan recently released a book called “Resisting AI” that sounds quite interesting. I heard him interviewed about it on a recent episode of Tech Won’t Save Us and enjoyed it.

I thought Murray Shanahan’s “Talking About Large Language Models” was good: https://arxiv.org/pdf/2212.03551.pdf

What is Bob, a representative human, doing when he correctly answers a straightforward factual question in an everyday conversation? To begin with, Bob understands that the question comes from another person (Alice), that his an- swer will be heard by that person, and that it will have an effect on what she believes. In fact, after many years together, Bob knows a good deal else about Alice that is relevant to such situations: her background knowledge, her interests, her opinion of him, and so on. All of this frames the communicative intent behind his reply, which is to impart a certain fact to her, given his understanding of what she wants to know.

Moreover, when Bob announces that Burundi is to the south of Rwanda, he is doing so against the backdrop of various human capacities that we all take for granted when we engage in every- day commerce with each other. There is a whole battery of techniques we can call upon to ascertain whether a sentence expresses a true proposition, depending on what sort of sentence it is. We can investigate the world directly, with our own eyes and ears. We can consult Google or Wikipedia, or even a book. We can ask some- one who is knowledgeable on the relevant subject matter. We can try to think things through, rationally, by ourselves, but we can also argue things out with our peers. All of this relies on there being agreed criteria external to ourselves against which what we say can be assessed.