Stwawberry
I instinctively read that in Homestar Runner’s voice.
“Appwy wibewawy!”
“Dang. This is, like… the never-ending soda.”
“Ah-ah, ahh-ah, ahhh-ahhh…”
Welp time to spend 3 hours rewatching all the Strongbad emails.
the system is down?
Strawbery
Strawbery
Strawbery
stawebry
Strarbey
The T in “ninja” is silent. Silent and invisible.
“Create a python script to count the number of
rcharacters are present in the stringstrawberry.”The number of 'r' characters in 'strawberry' is: 2
You need to tell it to run the script
Welp, it’s reached my level of intelligence.
Aww, C’mon, don’t sell yourself short like that, I’m sure you’re great at… Something…
For example, you would probably be way more useful than an AI, if there was a power outage.
Geee, you really mean that?!
Sure, when the chips fall, eating a computer rig won’t stave off starvation for even a minute.
O.O
Many intelligences are saying it! I’m just telling it like it is.
This is hardly programmer humor… there is probably an infinite amount of wrong responses by LLMs, which is not surprising at all.
I don’t know, programs are kind of supposed to be good at counting. It’s ironic when they’re not.
Funny, even.
Eh
If I program something to always reply “2” when you ask it “how many [thing] in [thing]?” It’s not really good at counting. Could it be good? Sure. But that’s not what it was designed to do.
Similarly, LLMs were not designed to count things. So it’s unsurprising when they get such an answer wrong.
the ‘I’ in LLM stands for intelligence
I can evaluate this because it’s easy for me to count. But how can I evaluate something else, how can I know whether the LLM ist good at it or not?
Assume it is not. If you’re asking an LLM for information you don’t understand, you’re going to have a bad time. It’s not a learning tool, and using it as such is a terrible idea.
If you want to use it for search, don’t just take it at face value. Click into its sources, and verify the information.
deleted by creator
Isn’t “Sphinx of black quartz, judge my vow.” more relevant? What’s all the extra bit anyway, even before the “z” debacle?
5% of the times it works every time.
You can come up with statistics to prove anything, Kent. 45% of all people know that.
I was curious if (since these are statistical models and not actually counting letters) maybe this or something like it is a common “gotcha” question used as a meme on social media. So I did a search on DDG and it also has an AI now which turned up an interestingly more nuanced answer.
It’s picked up on discussions specifically about this problem in chats about other AI! The ouroboros is feeding well! I figure this is also why they overcorrect to 4 if you ask them about “strawberries”, trying to anticipate a common gotcha answer to further riddling.
DDG correctly handled “strawberries” interestingly, with the same linked sources. Perhaps their word-stemmer does a better job?
Lmao it’s having a stroke
many words should run into the same issue, since LLMs generally use less tokens per word than there are letters in the word. So they don’t have direct access to the letters composing the word, and have to go off indirect associations between “strawberry” and the letter “R”
duckassist seems to get most right but it claimed “ouroboros” contains 3 o’s and “phrasebook” contains one c.
DDG’s one isn’t a straight LLM, they’re feeding web results as part of the prompt.
“it is possible to train 8 days a week.”
– that one ai bot google made
Probably trained on this argument.
I bust out laughing when I got to here:
Ah, trained off that body builder forum post about days of the week I see.
Ladies and gentlemen: The Future.
Q: “How many r are there in strawberry?”
A: “This question is usually answered by giving a number, so here’s a number: 632. Mission complete.”
A one-digit number. Fun fact, the actual spelling gets stripped out before the model sees it, because usually it’s not important.
There ARE two "R"s in strawberry.
There’s also a third one, but you can’t have three without having two.
That reminds me, I have 1 finger. I also have two fingers, 3 fingers and all the way up to 10 fingers!
True fact.
It can also help you with medical advice.
Jesus hallucinatin’ christ on a glitchy mainframe.
I’m assuming it’s real though it may not be but - seriously, this is spellcheck. You know how long we’ve had spellcheck? Over two hundred years.
This? This is what’s thrown the tech markets into chaos? This garbage?
Fuck.
I was just thinking about Microsoft Word today, and how it still can’t insert pictures easily.
This is a 20+ year old problem for a program that was almost completely functional in 1995.
