Companies are training LLMs on all the data that they can find, but this data is not the world, but discourse about the world. The rank-and-file developers at these companies, in their naivete, do not see that distinction…So, as these LLMs become increasingly but asymptotically fluent, tantalizingly close to accuracy but ultimately incomplete, developers complain that they are short on data. They have their general purpose computer program, and if they only had the entire world in data form to shove into it, then it would be complete.
You must log in or # to comment.
That’s easy. The people profiting from it are pushing it hard.
And other companies who had something half-baked just threw it out to both say “me too!” and to ingest as much user input training data in order to catch up.
That’s why “AI” is getting shoved into so many things right now. Not because it’s useful but because they need to gobble up as much training data as they can in order to play catch up.
It’s this up and coming techbro generation’s blockchain
The actual off the walls sci-fi shit this tech could maybe be capable of (an AI “third hemisphere” neural implant that catches the human mind up to the kind of mass calculation that it falls behind traditional computing on) is so far removed from what’s currently supportable on commercial tech that it’s all entirely speculation.
this data is not the world, but discourse about the world
To be fair, the things most people talk about are things they’ve read or heard of, not their own direct personal experiences. We’ve all been putting our faith in the accuracy of this “discourse about the world”, long before LLMs came along.
Indeed. I’ve never been to Australia. I’ve never even left the continent I was born on. I am reasonably sure it exists, though, based on all the second-hand data that I’ve seen. I even know a fair bit about stuff you can find there, like the Crow Fishers and the Bullet Farm and the Sugartown Cabaret.
deleted by creator
If you are interested there is no direct evidence that Shakespeare ever went to Italy, but he knew plenty of people who did, and travel guides were popular at the time. 13 of his plays are at least partially set in Italy. So about 1/3rd.
Pretty impressive.
Companies are training LLMs on all the data that they can find, but this data is not the world, but discourse about the world.
I mean, the same can be said for your own senses. “You” are actually just a couple of kilograms of pink jelly sealed in a bone shell, being stimulated by nerves that lead out to who knows what. Most likely your senses are giving you a reasonably accurate view of the world outside but who can really tell for sure?
So, as these LLMs become increasingly but asymptotically fluent, tantalizingly close to accuracy but ultimately incomplete, developers complain that they are short on data.
Don’t let the perfect be the enemy of the good. If an LLM is able to get asymptotically close to accurate (for whatever measure of “accurate” you happen to be using) then that’s really super darned good. Probably even good enough. You wouldn’t throw out an AI translator or artist or writer just because there’s one human out there that’s “better” than it.
AI doesn’t need to be “complete” for it to be incredible.
I do sorta see the argument. We don’t fully see with our eyes, we also see with our mind. So the LLM is learning about how we see the world. Like a scanner darkly hehe.
Not really sure how big of a deal this is it or even if it is a problem. I need to know what the subjective taste of a recipe is, not the raw data of what it is physically.
this data is not the world
i think most ML researchers are aware that the data isn’t perfect, but, crucially, it exists in a digestible form.
Self-proclaimed luddite? Well that’s certainly a choice lol. Have fun being perpetually confused and angry I guess.
Yes, this type of luddite: https://thenib.com/im-a-luddite/
It’s just history repeating itself. We’ve seen dotcom bubble, web 2.0 rush, nft fiasco and now AI. All fueled by VC greed. On top of that in today’s world companies try to get as much data as possible and so called AI is the perfect tool for that.
Two things IMO
-
Your data, more of it.
-
Using power, mostly to justify the existence of their giant machinery, the Cloud. These machines are used because they HAVE to be used. Bitcoin, nft, now ai everything.
-
