One of the most heated discussions occurring on X at the moment is about the ethics of a GitHub project in which a person is running Saw-like “torture” and “pain” experiments on a series of locally hosted large language models, causing a series of effective altruists and people who believe LLMs are sentient to beg GitHub to delete the project on the grounds that the AI is suffering and that this glorified text adventure game is somehow cruel. The saga is an outgrowth of several recent viral papers and blog posts that have sparked a wildly tiresome conversation about AI consciousness and the idea of “model welfare,” which is essentially worrying about the “mental health” of AI bots and agents.
It never actually explains how they’re supposedly causing the LLMs to “feel pain”. They just want us to take their word for it that they’re torturing them.
It says they’re using a pain “signal” but what is that even supposed to mean? Are they prompting the LLM with a prompt like “this signal makes you feel pain”? All that would do is have it output language that would be appropriate for a situation like that, just like with any other prompt like “you are a travel agent”. And the website it links to with the experiment dashboard doesn’t show any text output from any of the models or where they got those examples.
So none of this makes any sense at all–what is “it” that would be “feeling” this “pain” ? Unless someone can explain exactly how it “works”, it’s just more hyped up bullshit from people trying to get attention.