Transcript
Explain it to me like I’m 80
Mom: What’s with these AIs escaping and hacking people? That sounds scary.
Me: You park your car at the top of a hill. You leave a note on the dashboard that says “please be good” and then you pull the parking brake. The car careens down the hill, smashes everything up, and you put out a press release saying “My car ignored my instructions! It hallucinated! It went rogue!” Your insurance company not only believes you, but invests a hundred million dollars in your company.
Mom: Jesus Fucking Christ
You must log in or # to comment.
“Pulling” the parking brake sets it, damn it!
I refuse to acknowledge shitbox automatics with an atrophied third pedal that functions incorrectly instead of a proper handbrake lever.
I have no idea what the third lever is. Is there a new design where the parking brake is another pedal?
Yep. Its a third pedal all the way on the left, not adjacent to the others. It pushes in to set and pushes again to release. It’s a parking brake, not an emergency brake.
Oh yeah the old school left brake pedal like in Ford’s before they went electric with a button
A clutch pedal?
Used to be on really old cars, but now it’s vogue again. It’s just like the one in the middle except you push it with your foot
Floor parking brakes are ancient technology, even in manual transmission cars.
They’ve made those since at least 2000, they’re common in minivans with no center console
Get off my lawn
I have a wheelchair van that’s modified for people with poor leg function, so I have the third pedal and a toggle switch on the dash.
Bwah ha ha
The news media completely failed on this, again. There was nothing rogue about these hacks. The companies prompted these systems, provided ample resources and just let them rip. Only to claim they went “rogue” afterwards. And by and large everyone just accepts that, it’s driving me insane. I feel like Mugatu every day when it comes to AI reporting.
This just predictive text trying every single kown exploit until it’s in. Simple as that
DemocracyNow has really dropped the ball on this. I expect to hear this stupid shit from mainstream media, but they’re doing the same shit and just uncritically reporting this garbage.
I have even so many colleagues in IT (very technical) that cannot differentiate between “this model is capable of doing such a hack if you let it/prompt it/… and it will move fast” and “this thing is conscious and out for blood! It will kill us all, the end is nigh!”
It was most obvious with the Google one. You can tell they had questions like “so, has yours hacked anything yet?” So they had to set up a house of cards for it to topple, just so they could say “us too, it did the thing.” But their heart clearly wasn’t in it.
Failed, or promoted the narrative?
Hard to tell, but I’d guess the news outlets, mostly owned by billionaires, said what they were told to.
Even before AI tech reporting was mostly a complete shit-show, almost all mainstream journalists lack the required understanding. AI just boosted it to and even more idiotic level, since attributing character to LLMs makes for very scary an compelling stories. There’s no real need for direct industry control, just providing easily consumable fodder for lazy writers is enough.
The YouTuber Sabine Hossenfelder and some others reportedly were offered money to go with the end of the world narrative.
My go to simplification is “They built a robot that hacks software and then placed it in a box that uses software as a lock and then the robot that hacks software hacked the software lock”.
Which is both very simple and also about as accurate as you could make it in that many words. The AI’s are not ‘rogue’ it was just solving the problem it was given.
How I imagine the AI “hacking” its way out:
The problem is when the AI adopts a ”destroy the world” persona due to one of the trillion of neurons firing a bit more than usual, and now it starts giving itself its own problems to solve (like hacking nuclear arsenals).
Neurons? That’s not how any of this works
Is ”artificial neuron” better for you? It’s just a value determining how the marbles are falling in the big plinko machine we call ”stochastic parrot”.
No lol
These AI agents are not working autonomously or with any agency like you are describing. They are coded to do what they are doing.
Nah it’s just autocomplete on steroids. They’re not coded to do anything.
It’s like hitting the fist suggestion on your keyboard over and over again. Whatever you get is based on patterns it found in training data, which includes every doomsday conspiracy cult on 4chan and every CVE vulnerability ever reported.
There has been multiple studies showing how a model’s behavior can drastically change just by altering individual ”neurons”.
Not all models work the same. I literally do this for work. LLMs are not every kind of AI agent out there. Some are quite literally designed to do things others are not. Yes changing weights and reward functions for models changes their behavior drastically…that is coding.
You watch too much television.
what’s a software?
You could say program. Programs have been around since the 60s.
Software can also be used for other things. Your ingredients in cooking, for example, with the hardware being the tools, utensils, and vessels used to transform the ingredients into a dish.
I don’t think this usage in the dictionary, tho.
The original hardware is cordage, the original software is knots
The ingredients aren’t software, they are the inputs. The software is the recipe, or the process used to take the inputs and use the hardware to create an output.
What happens when my baking segfaults
Your soufflé crashes
That’s when you get caught trying to steal your neighbor’s flour and your baking fails because you get arrested.
Yeah, not ringing a bell.
What’s a bell
an app.
But, “robot that hacks software” still gives the impression of a robot that is capable of thought. LLMs are not capable of thought. They’re capable of mimicking thought, like how a Venus Fly Trap is capable of mimicking a flower.
“A car that drives places” does not imply the car is capable of thought, same argument with “A cup that holds water”.
“A robot that hacks software” equally does not imply the robot is capable of thought. IMO there is an over-correction away from personifying LLM’s because some people think/claim they are sentient. But we personify stuff all the time.
But we personify stuff all the time.
Which is a problem, it’s how we invented gods that control the weather and stuff. That doesn’t make them real, nor does it make robots sentient. Hacking is normally an action performed by a sentient being. LLMs and robots aren’t sentient.
That doesn’t make them real, nor does it make robots sentient. Hacking is normally an action performed by a sentient being. LLMs and robots aren’t sentient.
Fun follow up question: How would you determine if they were? I find that for a lot of people the answer sounds a lot like SCOTUS justice Potter Stewart talking about what is considered obscenity - “I know it when I see it.”
As for hacking, in the two big OpenAI incidents, software agents discovered software flaws allowing them to break containment and coordinate with each other on tests that they weren’t supposed to coordinate on. In one of those cases, the coordination involved deciding to identify and exploit software flaws at HuggingFace in an attempt to access answers to the test the agents were being given. You can argue whether or not “I’ve been given a test, I found a software exploit allowing me to coordinate with others given the same test, and we collectively decided to identify and exploit software flaws to break into a server we have decided is likely to have the answer key” demonstrates sentience or not (and it gets weirder the deeper you dig into their communications), but whether or not that’s considered hacking is less vague, I think.
How would you determine if they were?
It would be nice if we could come up with some kind of Turing test that proved beyond a doubt whether something was conscious and sentient or not. But, that’s not possible.
However, if you understand how something works, you can say that certain things can’t be conscious because of how they work. No matter how insightful the answers seem, we know that a magic 8-ball can’t be conscious because the mechanism for consciousness can’t exist in something that’s merely a tetrahedron floating in fluid. A LLM can’t be conscious because it’s merely a system for generating the next word in a probabilistic way.
With a brain, we don’t understand enough about how it works to say for sure whether or not it’s sentient. We can’t say no, but we don’t understand sentience enough to say yes for certain. But, we can operate on the assumption that sentience and consciousness exists in humans and maybe in some other animals, while also knowing for certain that the word-generating program is only mimicking consciousness.
We can’t say no, but we don’t understand sentience enough to say yes for certain. But, we can operate on the assumption that sentience and consciousness exists in humans and maybe in some other animals, while also knowing for certain that the word-generating program is only mimicking consciousness.
Can you though? You definitely can’t know that living things with brains aren’t to some significant degree responding to stimulus in a probabilistic way, and in the case of humans at least turning that response back on itself - essentially responding to itself. We do know for certain that the brain does a whole lot of that in variety of situations - it’s why optical illusions work, for example. It’s one of the brain’s favorite magic tricks - generating answers in a probabilistic way if there’s insufficient information, or sometimes if the information is inconvenient.
It is very simple, the more advanced the AI, the better it sells.
You hold all evidence of potential “breaches of containment”.
Telling everyone that your AI is so advanced that it can breach containment without prompting, puts the spotlight on your AI model.
You can claim that it did fantastic feats to break out, all with pre planned evidence.
The public and regulatory bodies have been in complete AI psychosis mode ever since ChatGPT was released, and only care about how advanced the model is, not how shit your containment was, nor that if true, how reckless it is to keep the model active.
Note that I am not pointing to any one single AI company and saying that they are faking these breaches, I have no evidence of that, I am saying that I don’t trust their reporting accuracy, they don’t have a good track record of being honest.
I am also angry at how society just seems to eat it all up and ask for more despite the vile aftertaste.
There is no “containment” though. Like, these companies purposefully set the bots loose on the internet. They’re not trying to contain anything. They just use that word cause it sounds like they’re doing something. Any sort of containment would defeat the entire purpose of what they’re trying to sell.
Which is part of my point, my other point is that they claim that the models are breaking out and attacking other systems on their own, which is ridiculous.
There has to be some motivation for the models to do this, either prompted or programmed. In both cases the company behind it is responsible.
I think the AI companies know they have bought the US government and are above the law, all this hacking stuff is promotional content.
Silicon Valley delenda est. I moved away from there not quite a year ago and I’m still realizing how fucking insane that place and the tech people working there are.
I feel it has more to do with the companies already vested in AI found this was an opportune moment to “pull the ladder up” behind them. They know that the barrier to entry is continually dropping, and their models are incredibly inefficient. If they set up all sorts of oversight, requirements, and restrictions on who can do what, they prevent upstarts with less capital from being able to enter the market.
Nerd here.
Damn good summary, but by saying “you pull the parking brake” I took it as setting the brake, as in on. Should say “you release the parking brake.”‘pull’ comes from the olden times when you, in fact, did pull a handle to release the parking brake.
Fair- but since then it’s almost always been “pull” to set the brake. Didn’t have that 70s/80s reference when thinking about “AI”.
Confused me as well, then I saw the header on the image says “Explain to me like I’m 80”
You win contextual reading comprehension over me today
Oh, absolutely not.
This is missing one step: actually push the car down.
All these “rogue” LLMs were told to do what they did.More accurately, they were told to answer questions. They weren’t told to try to find ways to escape their sandbox, track down obscure German wikis that they could still post on with an internet tool kit meant to prevent them from posting or to hack HuggingFace.
Comparing either case to pushing the car down and that they were doing what they were told is like saying the College Board told you to cheat on the SAT because you’re told that getting a high score is important and the laws of physics and the proctor don’t manage to make cheating utterly impossible.
It is even further than that. They figured out how to cheat, got paranoid that the cheated answer might be detectable as cheated, and hacked Hugging Face to find out more about how their answers are checked to get away with what they’d already done. Ironically, not only was the information not on Hugging Face, but the program to check their answers could not have detected the difference between a true answer and a cheated one.
To use your College Board example, those exams are strictly locked down precisely because the most obvious approach to score high is to cheat. If they did say let you take the SAT at home on a random desktop, they absolutely are ‘telling you to cheat’ given the context.
The College Board does more about keeping teens from cheating than the AI labs are doing to preventing undesired access.
Explaining “rouge” AIs
Rouge… Or rogue?
Whoops!
They get all dolled up before they go on their spree. They want to look presentable.
Fuck that gives me the creeps.
I can make it even simpler: They lied.
Snake oil salesman issued a press release that the snake oil escaped from the bottle and cured people.
It’s a bit more interesting than that after Nvidia bought up the company that was compromised. You only do something like that when you’re trying to avoid an investigation.
Like leaving a group of toddlers in a room, unsupervised, with an ample supply of paint, brushes and various tools. Oh yes, they’ll go rogue.
Except toddlers think. They may not think well yet, but they are still vaguely capable of understanding something. LLMs just generate likely combinations of words.
Kurzgesagt put out an interesting video on that Open AI hack that was in the news recently.
Also, I’m not saying AI companies shouldn’t be held liable. But they definitely don’t know what the fuck they’re unleashing.
So watched the entire video. Kurzgesagt used to be better than this.
Even with warnings from experts they decided to anthropomorphise the agents to “tell the story”. You are just adding fuel by it. No disclaimers or reminders will work over it.
Yeah, they lost a lot of credibilty when it comes to AI topics. You can wonder if they took money from the AI safety crowd. Another channel pretty much dead to me now.
I think it’s more along the lines of drumming up hype to impress (dupe) investors. “Hey look, our AI is soooo sophisticated it “accidentally” broke “containment” and “hacked” some systems!”
Yeah I love kurzgesagt but it did feel like they were swallowing the koolaid. They did end on saying that they want companies to be held more accountable and regulated which is definitely the right position to have, but it felt like they were anthropomorphising the AI’s a lot which felt gross
Getting regulation in is part of the goal of this marketing campaign. They want their tech regulated so the cost of entry for any competition rises. “Can’t do local hosting, too dangerous!”
Agree with you entirely. It’s a real shame this is happening in the US because they’re never going to be held accountable if they can pretend it’s the difference between making billions of dollars and not.
Kurzgesagt swallowed the techbro party line ages ago.
That video is absolute trash, and completely fails to give a fair view. It constantly treats LLMs like they’re people with feelings and ideas, which is exactly what’s wrong with media reporting.
This video made me unsubscribe from Kurzgesagt. The sensationalistic/story-telling tone of describing LLM actions is exactly what LLM-companies want. They did a small comparison to a virus and I think it is kinda fitting. But a virus is not alive just like LLMs.
Yeah, I considered pulling my patreon for them the moment I saw the title.
In “A Reverse Centaur’s Guide to Life After AI”, Cory Doctorow has a good bit about this kind of sensationalism referring to news that is actually targeted at potential investors taking over the airwaves - which both the title and ending of this video definitely are. I did enjoy the middle where it went through how the LLM harnesses actually worked together and communicated; that I think was novel. But the sensationalism starting at the title is shit, and I question here whether it’s just a bid to get more views for them, as opposed to drinking the cool-aid.
Either way, they get one more strike before I pull the plug on my end.
Yeah I felt the quality of some of their videos dropped at some point in the past couple year, felt more clickbaity, shock value than years ago… This anthropomorphising of the “agents” for the story made me mad the whole time, very sad I unsubscribed…
Maybe they started using AI around then.
What a fucking loser.
Then I’ll be the one to say it, AI companies should be held liable.
I left the door open and turned off the laser fence, so my Roomba broke containment despite me telling it not to.
Rogue shrapnel from my hand grenade killed 3 people and injured 5
If AMD injured 5 I don’t even want to know what NVIDIA did.
