causing a series of effective altruists and people who believe LLMs are sentient to beg GitHub to delete the project on the grounds that the AI is suffering
This coming from the same people who argue "Any suffering and death we inflict on humanity today, developing technology, will be offset by the suffering and death that technology might prevent in the future.
These people legit consider LLMs to be more human than humans.
Garbage in garbage out.
For me it’s not about the LMMs. It doesn’t matter if they‘re conscious or not, important is that they‘re simulating somewhat human behaviour.
I‘m cautious about someone who tortures simulations and says „Haha, it doesn’t matter they‘re not conscious!!!“
LLMs are not simulating human behaviour, they are stochastic text models. A magic text microwave so to speak. If you ask an LLM “What is 2+2” it does not know that it’s 4 and it does not even know what math is. The implicit prompt, so to speak, is “What is the most probable text following 2+2”, which is “=4”. This is what makes the responses so uncanny-valley-human-like: it’s a distilled stochastic model of how humans communicate, but since it’s stochastic over all possible human data, this is what pushes it into uncanny valley territory and is what we usually register as “huh, is this AI”, and this is also what makes anthropomorphism so easy. In the same way LLMs have data on human responses to discomfort, pain, torture, which is what they regurgitate in this “experiment”. All this is a overengineered way of telling the LLM to scream “ouch oof owie my ram” but like a human.
Have you never played the Sims?
Good point, lol. Still, I always felt kinda bad if something bad happened to them.
I have to question the morality of anyone who would even take the time to setup an LLM torture scenario.
I have no trouble with the idea of intentionality trying to confuse and confound an LLM to protect my privacy, but I don’t know that I would ever think of it in terms of torture for the LLM and thinking of it in that way seems a bit deranged.
We already know that habitually using an LLM will weaken your critical thinking.
It would not be surprising to learn that habitually abusing an LLM would weaken your empathy.