I’m usually the one saying “AI is already as good as it’s gonna get, for a long while.”

This article, in contrast, is quotes from folks making the next AI generation - saying the same.

  • hendrik
    link
    fedilink
    English
    14
    edit-2
    30 days ago

    Though, I don’t think that means they won’t get any better. It just means they don’t scale by feeding in more training data. But that’s why OpenAI changed their approach and added some reasoning abilities. And we’re developing/researching things like multimodality etc… There’s still quite some room for improvements.

    • @MajorHavoc@programming.devOP
      link
      fedilink
      English
      530 days ago

      Though, I don’t think that means they won’t get any better. It just means they don’t scale by feeding in more training data.

      Agreed. There’s plenty of improvement to be had, but the gravy train of “more CPU or more data == better results” sounds like it’s ending.

  • Tux
    link
    fedilink
    English
    1030 days ago

    Looks, like AI buble is slowly coming to end just like what happned to crypto and NFT buble.

    • Rikudou_Sage
      link
      fedilink
      English
      1530 days ago

      Sure, except for the thousands of products working pretty well with current gen. And it’s not like it’s over, now we’ve hit the limit of “just throw more data at the thing”.

      Now there aren’t gonna be as many breakthroughs that make it better every few months, instead there’s gonna be thousand small improvements that make it more capable slowly and steadily. AI is here to stay.

      • @Telorand@reddthat.com
        link
        fedilink
        English
        1129 days ago

        The bubble popping doesn’t have to do with its staying power, just that the days of, “Hey, I invented this brand new AI that’s totally not just a wrapper for ChatGPT. Want to invest a billion dollars‽” are over. AGI is not “just out of reach.”

      • @MajorHavoc@programming.devOP
        link
        fedilink
        English
        128 days ago

        The bubble was when we were being sold block chain as the solution to every problem. I feel like that bubble ended in 2019 or 2020.

        Things that actually benefitted from block chain are still around, of course.

        Unrelated side rant: I’m pissed about pogs going away, though. Pogs were fun. I should still be able to buy pogs.

  • @ChicoSuave@lemmy.world
    link
    fedilink
    English
    7930 days ago

    I understand folks don’t like AI but this “article” is like a reddit post with lots of links to subjects which are vague and need the link text to tell us what is important, instead of relying on the actual article.

    • @ggtdbz@lemmy.dbzer0.com
      link
      fedilink
      English
      229 days ago

      I see a lot of links here and there to this domain but I haven’t really read anything from there. I’m literally just scrolling through these comments to see if anyone has a comment like yours.

      My impression was that it’s just a blog but you calling it “a reddit post” is also interesting. What’s with this site? It looks like a decent amount of people think these takes are interesting. I have to deal with a lot of management people who love AI buzzwords, so a whole blog just ripping into it really speaks to me.

    • @11111one11111@lemmy.world
      link
      fedilink
      English
      43
      edit-2
      30 days ago

      What the fuck you aren’t kidding. I have comment replies to trolls that are longer than that article. The over the top citations also makes me think this was entirely written by an actual AI bot that was lrompted to supply x amoint of sources in their article. Lol

    • @Korne127@lemmy.world
      link
      fedilink
      English
      2428 days ago

      LLMs are one version of AI. It’s just one tiny part of AIs that are used every day, from chess bots to voice transcription, but they also are AI.

        • @FiskFisk33@startrek.website
          link
          fedilink
          English
          228 days ago

          Of course it changes meaning if you remove the qualifier.

          Artificial

          Adjective

          1. artificial (comparative more artificial, superlative most artificial)
            Man-made; made by humans; of artifice.
            The flowers were artificial, and he thought them rather tacky.

          2. Insincere; fake, forced or feigned.
            Her manner was somewhat artificial.

          In effect, man-made/fake intelligence.

      • @buddascrayon@lemmy.world
        link
        fedilink
        English
        8
        edit-2
        28 days ago

        I would replace the word version with aspect. LLMs are merely one part of the puzzle that would be AI. Essentially what’s been constructed is the mouth and the part of the brain that can form words but without any of the reasoning or intelligence behind what the mouth says.

        The same goes for the art AIs. They can paint pictures based on input but they can’t reason how those pictures should look. Which is why it requires so much tweaking to get them to output something that doesn’t look like it came out of a Lovecraft novel.

      • @raspberriesareyummy@lemmy.world
        link
        fedilink
        English
        126 days ago

        Not at all. AI is something that uses rules, not statistical guesswork. A simple control loop is alreadu basic AI, but the core mechanism of LLMs is not (the parts before and after token association/prediction are). Don’t fall for marketing bullshit of some dumbass silicon valley snake oil vendors.

  • @nialv7@lemmy.world
    link
    fedilink
    English
    229 days ago

    They might be right but I read some of the linked articles on this blog (?), the authors just come off as not really knowing much about current AI technologies, and at the same time very very arrogant.

    • @raspberriesareyummy@lemmy.world
      link
      fedilink
      English
      128 days ago

      The article talks about LLM developers / operators. Not sure how you got from that to “current AI technologies” - a completely unrelated topic.

  • @daniskarma@lemmy.dbzer0.com
    link
    fedilink
    English
    4229 days ago

    It’s pretty obvious that they will hit a ceiling.

    Quick buck is over. And now it’s time again for base research to create better approach.

    I really wish we had a really advanced AI with reasonable resource consumption within my lifetime. I don’t think it’s unreasonable as we have got really far in the last 30 years of computational technology.

    • @raspberriesareyummy@lemmy.world
      link
      fedilink
      English
      1828 days ago

      I really wish we had a really advanced AI with reasonable resource consumption within my lifetime.

      You only wish that for as long as it doesn’t happen. Have you looked at the world we live in? Such tools would be controlled by the same billionaire dipshits for their personal gain as all social media is being used already.

    • @buddascrayon@lemmy.world
      link
      fedilink
      English
      628 days ago

      The problem isn’t with the AI. It’s with how it’s being treated. It’s currently being sold as if it were general intelligence. Which it’s not. It should instead be treated like it’s a mindless tool. Something that is inert on its own. Useful for some things but only in a limited sense. Unfortunately the companies, who have spent millions of dollars developing these things, are trying to sell it as the “do-all” artificial intelligence that people have grown up seeing in sci-fi media. Which it 100% is not.

      • @daniskarma@lemmy.dbzer0.com
        link
        fedilink
        English
        2
        edit-2
        28 days ago

        Every company have always oversell their own products. This is not new.

        Coca Cola is also just a carbonated sweet drink and it’s being sold as happiness, socialization and the meaning of Christmas in a bottle.

        Companies oversell, it’s called marketing. It’s shit practice but it’s not nothing new.

        That does not make the technology worse (or better). Current AI technology has its uses. With a big problem in how resource hungry it is. But it’s fairly useful.

        • @buddascrayon@lemmy.world
          link
          fedilink
          English
          3
          edit-2
          25 days ago

          Your point is valid. Companies do use marketing to sell their products by using lots of outrageous claims. And my problem isn’t really what the companies it’s mostly with the people who are buying that bullshit.

          P.S. your Coca-Cola example would have been better if you had reached back to their origins when they were sold as a tonic that cures just about everything. What they’re being sold as right now is just a soda and all of the current marketing around it is just nostalgia bait which everybody uses for everything especially around Christmas time.

    • Cethin
      link
      fedilink
      English
      1629 days ago

      We’ve come a long way in computing, but the computational power difference between a human brain and a computer is significant. LLMs were just a smart way to have computers learn pattern recognition. While important, it isn’t anything close to artificial general intelligence (AGI), which is what the term AI usually means.

      • @Homescool@lemmy.world
        link
        fedilink
        English
        529 days ago

        Yeah.   AI may grind for a while but hardly anyone has put the current stuff to work, yet.   We will be feeling the benefits of what is released right now for a decade to come.   I am working on a very rudimentary application that will use ML at work and it won’t come out for 12 more months, and it hardly does anything but make the most obvious decisions 10m times faster than I can.   But it’s going to fundamentally change our labor model.

        There are regular folks applying amazing technologies that go way beyond content generation.

        The tech may grind but the application of that tech is barely getting its feet and should run hard for a decade.

  • @cron@feddit.org
    link
    fedilink
    English
    10630 days ago

    It’s absurd that some of the larger LLMs now use hundreds of billions of parameters (e.g. llama3.1 with 405B).

    This doesn’t really seem like a smart usage of ressources if you need several of the largest GPUs available to even run one conversation.

      • @blackbelt352@lemmy.world
        link
        fedilink
        English
        6430 days ago

        It’s a lot. Like a lot a lot. GPUs have about 150 billion transistors but those transistors only make 1 connection in what is essentially printed in a 2d space on silicon.

        Each neuron makes dozens of connections, and there’s on the order of almost 100 billion neurons in a blobby lump of fat and neurons that takes up 3d space. And then combine the fact that multiple neurons in patterns firing is how everything actually functions and you have such absurdly high number of potential for how powerful human brains are.

        At this point, I’m not sure there’s enough gpus in the world to mimic what a human brain can do.

        • @cynar@lemmy.world
          link
          fedilink
          English
          2229 days ago

          That’s also just the electrical portion of our mind. There are whole levels of chemical, and chemical potentials at work. Neurones will fire differently depending on the chemical soup around them. Most of our moods are chemically based. E.g. adrenaline and testosterone making us more aggressive.

          Our mind also extends out of our heads. Organ transplant recipricants have noted personality changes. Food preferences being the most prevailant.

          The neurons only deal with ‘fast’ thinking. ‘slow’ thinking is far more complex and distributed.

      • @cron@feddit.org
        link
        fedilink
        English
        1830 days ago

        I don’t think your brain can be reasonably compared with an LLM, just like it can’t be compared with a calculator.

        • @GetOffMyLan@programming.dev
          link
          fedilink
          English
          2230 days ago

          LLMs are based on neural networks which are a massively simplified model of how our brain works. So you kind of can as long as you keep in mind they are orders of magnitude more simple.

          • @utopiah@lemmy.world
            link
            fedilink
            English
            629 days ago

            At some point it becomes so “simplified” it’s arguably just not the same thing, even conceptually.

            • @GetOffMyLan@programming.dev
              link
              fedilink
              English
              2
              edit-2
              28 days ago

              It is conceptually the same thing. A series of interconnected neurons with a firing threshold and weighted connections.

              The simplification comes with how the information is transmitted and how our brain learns.

              Many functions in the human body rely on quantum mechanical effects to function correctly. So to simulate it properly each connection really needs to be its own super computer.

              But it has been shown to be able to encode information in a similar way. The learning the part is not even close.

              • @utopiah@lemmy.world
                link
                fedilink
                English
                123 days ago

                It is conceptually the same thing. […] The learning the part is not even close.

                Well… isn’t the “learning part” precisely the point? I don’t think anybody is excited about brains as “just” a computational device, rather the primary function of a brain is … learning.

                • @GetOffMyLan@programming.dev
                  link
                  fedilink
                  English
                  123 days ago

                  No, we are nowhere close to learning as the human brain does. We don’t even really understand how it does at all.

                  The point is to encode solutions to problems that we can’t solve with standard programming techniques. Like vision, speech recognition and generation.

                  These problems are easy for humans and very difficult for computers. The same way maths is super easy for computers compared to humans.

                  By applying techniques our neurones use computer vision and speech have come on in leaps and bounds.

                  We are decades from getting anything close to a computer brain.

    • @WalnutLum@lemmy.ml
      link
      fedilink
      English
      1729 days ago

      Seeing as how the full unquantized FP16 for Llama 3.1 405B requires around a terabyte of VRAM (16 bits per parameter + context), I’d say way more than several.

  • Greg Clarke
    link
    fedilink
    English
    5430 days ago

    OpenAI, Google, Anthropic admit they can’t scale up their chatbots any further

    Lol, no they didn’t. The quotes this articles are using are talking about LLMs not chatbots. This is yet another stupid article from someone who doesn’t understand the technology. There is a lot of legitimate criticism for the way this technology is being implemented but FFS get the basics right at least.

    • @MajorHavoc@programming.devOP
      link
      fedilink
      English
      2930 days ago

      Are you asserting that chatbots are so fundamentally different from LLMs that “oh shit we can’t just throw more CPU and data at this anymore” doesn’t apply to roughly the same degree?

        • Greg Clarke
          link
          fedilink
          English
          429 days ago

          People that don’t understand those terms are using them interchangeably

      • Greg Clarke
        link
        fedilink
        English
        529 days ago

        Yes of course I’m asserting that. While the performance of LLMs may be plateauing, the cost, context window, and efficiency is still getting much better. When you chat with a modern chat bot it’s not just sending your input to an LLM like the first public version of ChatGPT. Nowadays a single chat bot response may require many LLM requests along with other techniques to mitigate the deficiencies of LLMs. Just ask the free version of ChatGPT a question that requires some calculation and you’ll have a better understanding of what’s going on and the direction of the industry.

        • @MajorHavoc@programming.devOP
          link
          fedilink
          English
          929 days ago

          I think you’re agreeing, just in a rude and condescending way.

          There’s a lot of ways left to improve, but they’re not as simple as just throwing more data and CPU at the problem, anymore.

          • Greg Clarke
            link
            fedilink
            English
            3
            edit-2
            29 days ago

            I’m sorry if I’m coming across as condescending, that’s not my intent. It’s never been “as simple as just throwing more data and CPU at the problem”. There were algorithmic challenges for every LLM evolution. There are still lots of potential improvements using the existing training data. But even if there wasn’t, we’ll still see loads of improvements in chat bots because of other techniques.

            Edit: typo

    • @Voroxpete@sh.itjust.works
      link
      fedilink
      English
      1129 days ago

      Claiming that David Gerrard an Amy Castor “don’t understand the technology” is uh… Hoo boy… Well it sure is a take.

      • Greg Clarke
        link
        fedilink
        English
        429 days ago

        The title of the article is literally a lie which is easily fact checked. Follow the links to quotes in the article to see what the quoted individuals actually said about the topic.

          • Greg Clarke
            link
            fedilink
            English
            329 days ago

            I know the difference. Neither OpenAI, Google, or Anthropic have admitted they can’t scale up their chat bots. That statement is not true.

            • @Voroxpete@sh.itjust.works
              link
              fedilink
              English
              528 days ago

              So is your autism diagnosed or undiagnosed?

              I ask this as an autistic person, because the only charitable way to read what’s happening here is that you’re clearly struggling with statements that aren’t intended to be read completely literally.

              The only other way to read it is that you’re arguing in bad faith, but I’ll assume thats not the case.

  • Ragdoll X
    link
    fedilink
    English
    7
    edit-2
    30 days ago

    It’s a known problem - though of course, because these companies are trying to push AI into everything and oversell it to build hype and please investors, they usually try to avoid recognizing its limitations.

    Frankly I think that now they should focus on making these models smaller and more efficient instead of just throwing more compute at the wall, and actually train them to completion so they’ll generalize properly and be more useful.