not reading the fucking sidebar

Yeah, I get that this is a place to vent. And I get why to vent about this. LLMs and other A"I" systems (with quotation marks because this shite is not intelligent!) are being shoved down every bloody where, regardless of actual usefulness, safety, or user desire. Telling you to put glue on your pizza, to eat poisonous mushrooms, that “cherish” has five letters, that Latin had no [w], that the Chinese are inferior to Westerners.

While a crowd of irrationals tell you “it is intelligent, you can’t prove otherwise! CHRUST IT YOU DIRTY SCEPTIC/INFIDEL/LUDDITE REEEE! LALALA I’M PRETENDING TO NOT SEE THE HALLUCINATION LALALA”.

I also get the privacy nightmare that this shit is. And the whole deal behind “we’re using your content as training data, and then selling the result back to you”. Or that it’s eating electricity like there’s no tomorrow, in a planet where global warming is a present issue.

I get it. I get it all. That’s why I’m here. And if you (or anyone else) think that I’m here for any other reason, by all means, check my profile - you’ll find plenty pieces of criticism against those stupid corporate AI takes from vulture capital. (And plenty instances of me calling HN “Redditors LARPing as Hax0rz”. )

However. Pretending that there’s no use case ever for LLMs is the wrong way to go.

and thinking this is high school debate club fallacy

If calling it “nirvana fallacy” rubs you the wrong way, here’s an alternative: “this argument is fucking stupid, in a very specific way: it pretends that either something is perfect or it’s useless, with no middle ground.”

The other user however does not deserve the unnecessary abrasiveness so I’ll keep simply calling it “nirvana fallacy”.

But the claims of the text are often why you read it in the first place!

By “not caring about the former” [claims], I mean in the LLM output, because you know that the LLM will fuck them up. But it’ll still somewhat accurately represent the topic of the text, and you can use this to your advantage.

You might as well just skim the titles and guess.

Nirvana fallacy.

(For clarity I’ll re-emphasise that my top comment is the result of misreading the word “documents” out, so I’m speaking on general grounds about AI “summaries”, not just about AI “summaries” of documents.)

The key here is that the LLM is likely to hallucinate the claims of the text being shortened, but not the topic. So provided that you care about the later but not the former, in order to decide if you’re going to read the whole thing, it’s good enough.

And that is useful in a few situations. For example, if you have a metaphorical pile of a hundred or so scientific papers, and you only need the ones about a specific topic (like “Indo-European urheimat” or “Argiope spiders” or “banana bonds”).

That backtracks to the OP. The issue with using AI summaries for documents is that you typically know the topic at hand, and you want the content instead. That’s bad because then the hallucinations won’t be “harmless”.

Every time that I start a new Thrive game, I tell myself “I’m going to try a different strategy”. And I always end with a wide and thin carnivorous plant, with energy being provided by chloroplasts and N+P being provided by eating whatever I find in my path.

No, it’s just rambling. My bad.

I focused too much on using AI to summarise and ended not talking about it summarising documents, even if the text is about the later.

And… well, the later is such a dumb idea that I don’t feel like telling people “the text is right, don’t do that”, it’s obvious.

You could use them to know what the text is about, and if it’s worth your reading time. In this situation, it’s fine if the AI makes shit up, as you aren’t reading its output for the information itself anyway; and the distinction between summary and shortened version becomes moot.

However, here’s the catch. If the text is long enough to warrant the question “should I spend my time reading this?”, it should contain an introduction for that very purpose. In other words if the text is well-written you don’t need this sort of “Gemini/ChatGPT, tell me what this text is about” on first place.

EDIT: I’m not addressing documents in this. My bad, I know. [In my defence I’m reading shit in a screen the size of an ant.]

Really my point is there are enough things to criticize about LLMs and people’s use of them, this seems like a really silly one to try and push.

The comment that you’re replying to is fairly specifically criticising the usage of the word “hallucination” to misrepresent the nature of the undesirable LLM output, in the context of people selling you stuff by what it is not.

It is not “pushing” another “thing to criticise about LLMs”. OK? I have my fair share of criticism against LLMs themselves, but that is not what I’m doing right now.

Continuing (and torturing) that analogy, […] max_int or small buffers.

When we extend analogies they often break in the process. That’s the case here.

Originally the analogy works because it shows a phony selling a product by what it is not. By making the phony to precompute 4*10¹² equations (a completely unrealistic situation), he stops being a phony to become a muppet doing things the hard way.

If it were the case that there had only been one case of a hallucination with LLMs, I think we could pretty safely call that a malfunction

If it happens 0.000001% of the time, I think we could still call it a malfunction and that it performs better than a lot of software.

Emphases mine. Those “ifs” represent a completely unrealistic situation, that does not show anything useful about the real situation.

We know that LLMs output “hallucinations” way more than just once, or 0.000001% of the time. They’re common enough to show you how LLMs work.

Rumors that Valve was working on a portable gaming unit had emerged in May 2021

Perhaps. The timing fits considerably better than the other alternatives mentioned until now.

True that.

I don’t rule out the possibility that COVID made some people think further on how they interact with software, and that indirectly promoted some Linux usage. However, I don’t think that it would create continuous pressure encouraging adoption, that keeps going on four years later.

Another reason why I don’t think that COVID is the cause is the timing: the “bulk” of the social impact happened in early 2020, but the slope seems to start near the end of 2020, almost early 2021.

To make it worse decision makers - regardless of country - are typically old and clueless about “this computer stuff”. As such they literally don’t see the problem.

“Suicide by AI” drinking game: drink a sip every time that a business gets rekt by trying to force LLM and/or diffusion models down the users’ throats.

I don’t think that it’s due to COVID, as it’s an upwards slope instead of just a spike, or spike + plateau.

What happened around '20?

I’m asking that because, the way that I’m reading this graph, there’s a plateau between '15 and '20, and then a slope upwards.

What’s “eternal” in “Eternal September” is not the influx of new users, but rather the disruption of the social norms caused by a huge and sudden influx of new users.

That disruption started in 1993, and never ended. So it had a beginning but no end as of yet.

It gets worse, when you remember that there’s no dividing line between harmful and healthy content. Some content is always harmful, some is by default healthy, but there’s a huge gradient of content that needs to be consumed in small amounts - not doing it leads to alienation, and doing it too much leads to a cruel worldview.

This is doubly true when dealing with kids and adolescents. They need to know about the world, and that includes the nasty bits; but their worldviews are so malleable that, if all you show them is nasty bits, they normalise it inside their heads.

It’s all about temperance. And yet temperance is exactly the opposite of what those self-reinforcing algorithms do. If you engage too much with content showing nasty shit, the algo won’t show you cats being derps to “balance things out”. No, it’ll show you even more nasty shit.

It gets worse due to profiling, mentioned in the text. Splitting people into groups to dictate what they’re supposed to see leads to the creation of extremism.

In the light of the above, I think that both Kaung and Cai are missing the point.

Kaung believes that children+teens would be better if they stopped using smartphones; sorry but that’s stupid, it’s proposing to throw the baby out with the dirty bathtub water.

Cai on the other hand is proposing nothing but a band-aid. We don’t need companies to listen to teens to decide what we should be seeing; we need them to stop altogether deciding what teens and everyone else should be seeing.

Ah, and about porn, mentioned on the text: porn is at best a small example of a bigger issue, if not a red herring distracting people from the issue altogether.

The point of Eternal September is that it happens all the time, so when was that?

Kind of - it doesn’t happen “all” the time; it has a beginning, but no end.

I agree that it’s wild. And it’s a bit bittersweet for me.

Usenet - and the old internet as a whole - were all about humans sharing stuff between themselves: I see something cool, I give you the link, you see something cool. While modern platforms try to remove the human from the equation, make them invisible: I see something cool, I “endorse” (upvote, like etc.) it, and that endorsement is used by some algorithm to automatically pick what you’re supposed to be seeing.

Reddit is both and neither at the same time. The links are manually picked and shared, like in the old internet; but they’re algorithmically sorted and ranked as in the new internet. It’s like a product of the old internet trying to carve its way into the new internet, but never completely ditching its roots.

Perhaps that’s why that site lasted so long. And I hope that one day we’re going to say “a shame that it died”.

I wouldn’t call pasting verbatim training data hallucination when it fits the prompt. It’s not necessarily making stuff up.

I’ve seen it being called hallucination plenty of times. Because the output is undesirable - even if it satisfies the prompt, it is not something you’d want the end user to see, as it shows that the whole thing is built upon the unpaid labour of everyone who uses the internet.

How would you call it? Only by their specific issues? Or would you use a general term, like “error” or “wrong”?

Calling the output by what it is (false, or immoral, or nonsensical) instead of a catch-all would be a progress, I think.

When it comes to the code itself you’re right, there’s no difference between “bug” and “not a bug”. The difference is how humans classify the behaviour.

And yet there’s a clear mismatch between what the developers of those large “language” models know that they’re able to do, versus what LLMs are being promoted for, and that difference is what is being called “hallucination”. They are not intelligent systems, the info that they output is not reliably accurate, it’s often useless rubbish. But instead of acknowledging it they label it “hallucination”.

Perhaps an example would be good here. Suppose that I made a text editor; it works nicely as a text editor and nothing much else. Then I make it automatically find and replace the string “=2+2” with “4”, and use it to showcase my text editor as if it was a calculator. “Look, it can do maths!”.

Then the user types down “=3+3”, expecting the “calculator” to output “6”, and it doesn’t. Can we really claim that the user found a “bug”? Not really. It’s just that I’m a phony and I sold him a text editor as if it was a calculator.

And yet that’s exactly what happens with LLMs.