• treadful@lemmy.zip
    link
    fedilink
    English
    arrow-up
    127
    ·
    12 days ago

    I still can’t decide if these people are delusional or I am.

    Every single time I use an LLM it fucking “lies” to me or otherwise completely fails at the task. The people talking like this seem to me like they’ve never actually used it, or haven’t actually vetted the accuracy (like most AI users).

    Maybe I’m just not using the “good stuff”. Or I’m not imaginative enough to foresee a near future where these problems are actually corrected and it becomes trustworthy.

    I’ve never been so torn by a technological prediction.

    • realitista@lemmus.org
      link
      fedilink
      English
      arrow-up
      32
      ·
      edit-2
      12 days ago

      I’m curious what you are using. The free versions of chatgpt have been like that for me, but even Gemini flash with extended thinking, also free for a while longer, is giving me pretty reliable results as long as there training data out there to derive an answer from. The higher (paid) Claude models will one shot most coding tasks.

      • tyler@programming.dev
        link
        fedilink
        English
        arrow-up
        25
        ·
        12 days ago

        Claude can one shot tasks until you get a larger system then it completely shits itself. These models are nothing more than autocomplete, and they can’t hold large systems in their heads. Anthropic literally tried to rewrite all of bun using Claude, they said they did it and yet it still hasn’t released six months later.

        • realitista@lemmus.org
          link
          fedilink
          English
          arrow-up
          1
          ·
          edit-2
          12 days ago

          Yeah I certainly wouldn’t advocate building a whole business around code it wrote. But for small personal tasks it hasn’t let me down. Building custom server applications, desktop applications, Firefox add-ons, upgrading my homeassistant 10 versions over a couple weeks without letting anything break. These sort of things it handles pretty easily and are all things I wouldnt get done without it.

      • treadful@lemmy.zip
        link
        fedilink
        English
        arrow-up
        22
        ·
        12 days ago

        I’ve not yet fucked with Claude. I don’t want to pay for it, and I really don’t like the surveillance aspect of these centralized systems. Mostly I’m using Gemini, whatever DDG had in their search results, and local models I’ve been fiddling with (like Qwen3.8 right now).

        All more or less garbage once I get into the details of anything on the edge of my expertise.

        • RyanDownyJr@lemmy.world
          link
          fedilink
          English
          arrow-up
          5
          ·
          12 days ago

          I’ve been using OpenCode with whateverthefuck free models they have listed on there and they all seem to do fine with agentic tasks like building me scripts or executables to make my work tasks easier.

          I used Gemini at the start with “Frontier Knowledge” and it seemed to do worse than the ones listed on OpenCode, but maybe that’s because i could only do like three prompts a week since I refuse to pay into an AI.

          end of the day, its just LLMs writing code for me, but I cannot see how this would be useful for a large scale code base, but also #NotAProgrammer.

        • realitista@lemmus.org
          link
          fedilink
          English
          arrow-up
          1
          ·
          12 days ago

          Are you using Gemini in flash extended thinking? (Not flash light) . I haven’t had many hallucinations other than cases where the training data it needs just doesn’t exist (cases where I can’t find the answers by googling either)

      • ThirdConsul@lemmy.zip
        link
        fedilink
        English
        arrow-up
        5
        ·
        11 days ago

        I mean as long as the coding tasks are simple, and seeded with enough context, then sure. As in “create tinder clone works”, but I still have to check the output in my company codebase. We are leveraging llms a lot for code writing, full agentic pipelines, loops, all the shiny new approaches.

        The outcomes in all cases are still mediocre or below.

        We have twice as many open prod bugs as a year ago.

      • Frettchen@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        9 days ago

        I use it professionally. You are talking to a toddler or a professional and you never know. It’s like an interview where the person tries to get a job but you want to use it in production.

    • jj4211@lemmy.world
      link
      fedilink
      English
      arrow-up
      11
      ·
      11 days ago

      Generally I’ve found that:

      If the facts are painfully obvious from a simple web search, then GenAI has a decent chance of getting it right. This can be useful if you can’t recall any “key” words well and the GenAI can craft several searches and get there.

      However, if it does mess up the facts, the result looks superficially the same as “correct”. So while it can give accurate data, you always have to double check. This can still be useful, as finding the right search terms can be a decent help.

      In coding, sometimes in some situations, you can have requirements that are absolutely testable, and thus you can have the models retry and retry until it works. This isn’t always feasible. Even when it seems feasible, you may screw up the criteria, or the GenAI when enough freedom disables a probablematic test rather than solve it, and it likely will generate code that’s not really fit to modify. There are a lot of situations where this is useful, but it is infuriating that non technical people and even some low skill technical people assume this is always the case.

      Then when you get away from facts mattering, it gets “better”. Example, someone jokingly asked for one to “make gta6”. After a while it came back with a GTA 1 clone. Lots of people were impressed, because whatever it did could be considered a success even as it obviously didn’t match the expectation. The operators also like to GenAI some webcomic, where it is a fiction. They almost always didn’t have any interesting thought going in so they tend to be crap, but “correctness” didn’t matter.

      Of course, also making fakes. Supreme case where looking correct matters but being factually correct does not matter at all. GenAI above all else “seems” correct.

    • kewjo@lemmy.world
      link
      fedilink
      English
      arrow-up
      10
      ·
      12 days ago

      from my point of view it feels like most people are willing to trade their cognitive function for being lazy, which is a boundary i never want to cross.

      AI will produce a lot of code but most of it is pretty poor quality as most training data is going to be poor quality code, there’s just always going to be more bad code than good to begin with just due to how difficult quality code really is to produce. I’ll give a hint, good code is usually small and succinct.

      since my work started pushing AI live site issues have increased dramatically. turns out the person using AI looks like they have a ton more productivity but in reality that just shifts to whoever is reviewing the code. and to those who will say its the developer’s responsibility to review the code, yeah no shit, but if you ever work corporate you realize most don’t care as long as their managers think they are productive and just blame others for being bottlenecks.

      Overall it just enlightened me to how bad the average developer really is, but i guess obtaining mediocrity is the sacrifice to make in the name of “productivity”.

    • fonix232@fedia.io
      link
      fedilink
      arrow-up
      9
      ·
      12 days ago

      Your experience is pretty unique then.

      Yes, LLMs make mistakes, but even small, self-hosted ones are pretty efficient today if you prompt them well. They’re not mind reading software so you need to be able to describe the task and HOW you want it done, not just barf in some basic instructions like “write me a copy of Facebook but better”.

      • treadful@lemmy.zip
        link
        fedilink
        English
        arrow-up
        5
        ·
        12 days ago

        No amount of prompt “engineering” will help when they outright make shit up.

        • fonix232@fedia.io
          link
          fedilink
          arrow-up
          3
          ·
          12 days ago

          Yes it does. Just need to go beyond prompt. Add reinforcement loops, make it test the solution in a separate environ it can’t screw up in, and have it not just INVENT things (“give me X”), but research the topic and base its solution on the rules created by the research.

          This is what basically the Claude harness (not the local but the remote harness you can’t see) adds to the LLM what makes it so powerful and useful. Replicate those processes and even a small 4B mode will be incredibly capable.

      • jtrek@startrek.website
        link
        fedilink
        English
        arrow-up
        4
        ·
        12 days ago

        I think most people are so disorganized in their thinking, they can’t “prompt” well. There’s a lot of unclarified assumptions and leaps in how many people communicate

        • fonix232@fedia.io
          link
          fedilink
          arrow-up
          1
          ·
          12 days ago

          And funnily enough, AI is great at helping streamline that process too. People just need to ASK for help (even if they’re asking the AI model) instead of being set in one way of thinking and expecting miracles.

      • WolfLink@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        4
        ·
        12 days ago

        Eh idk I still have it make mistakes constantly.

        Recent example: I asked AI to help me come up with a search/replace string for editing a file in vim. My prompt was something like “I am editing a text file in vim and I need to replace <description of pattern> with <description of pattern>. How can I do this with :s/ ?” And it spat out something that didn’t work. It was not valid syntax (for vim, I think it was valid standard regex, or close to it), and it didn’t quite follow the pattern I was trying to describe (although ofc that could be my fault to some extent). But it was useful in that it pointed me in the right direction to come up with a correct formula with some follow up googling and experimentation.

        That has been very typical of my experiences with AI. Useful sometimes, but absolutely not “does everything to the point you don’t have to think about it” that seems to be a common opinion.

        For context I’m using ~20GB local models, so not Claude, which people who pay for LLMs swear by.

        • Guy Ingonito@reddthat.com
          link
          fedilink
          English
          arrow-up
          2
          ·
          11 days ago

          I recently used it to customize a vimeopro library and I had to go through some back and forth with it but we did get there in under an hour.

          I have very little coding experience, I’m a graphic designer.

          I would’ve required the help of someone who’s salary would’ve been six figures to solve this problem without the Claude.

          So what it’s done is make computer code much cheaper than it previously was.

          • WolfLink@sh.itjust.works
            link
            fedilink
            English
            arrow-up
            1
            ·
            11 days ago

            Yeah its definitely useful for topics you personally don’t have much experience in. It’s much less useful when trying to use it for topics you personally are an expert in.

        • fonix232@fedia.io
          link
          fedilink
          arrow-up
          2
          ·
          12 days ago

          I use Claude + local models.

          Yes, older models will make those mistakes. The solution is to provide appropriate agent/skill definitions so it doesn’t just spit something out, but rather comes up with the solution, then smoke tests it in a separate scratch environment.

          Claude uses this approach for reinforced learning, and it works well. Takes a few more rounds to resolve, but the solution is generally flawless (for that specific purpose).

        • Kabaka@lemmy.blahaj.zone
          link
          fedilink
          English
          arrow-up
          2
          ·
          12 days ago

          This is basically the same as asking someone who knows a bit about everything to recite obscure information from memory. I doubt most humans would do better.

          What you’re missing here is a feedback loop, validation, skills, etc. For example, if I ask one of my well-configured agents the same question, they go read the help/man page/other docs, start a vim session, quickly test/iterate until the result is correct, then give me a one-page document explaining what I need to know, with cited evidence — far faster than I’d do it, and I can keep working for the few seconds it takes. This rigor is written into my global instructions, not something you get out of the box on most models (Anthropic’s models tend to be good at this without handholding, which is part of why they’re so popular, aside from the fact that they just don’t make as many mistakes).

          The same mindset scales to larger software problems, too. As long as you have a well-defined specification and good agent instructions (and/or something like Spec Kit), you can have agents break it down, implement, and then other agents compare the result to the spec, and just keep looping until it is done. The hard part is writing good specs and requirements, but that’s not a new problem.

          • WolfLink@sh.itjust.works
            link
            fedilink
            English
            arrow-up
            2
            ·
            11 days ago

            This is basically the same as asking someone who knows a bit about everything to recite obscure information from memory. I doubt most humans would do better.

            Sure, but a good web resource or even a good reference book would provide better help faster. Unfortunately it’s getting harder to harder to find those good web resources as search results get overtaken by AI slop.

            iterate until the result is correct, then give me a one-page document explaining what I need to know, with cited evidence — far faster than I’d do it

            I have gotten it to do this in certain situations, like the other day I wanted to simplify a math formula, so I gave it a loop with a Python script that checked its solution against the original reference version. This worked pretty well.

            But I had to write code specifically for that situation. Even with a good skeleton to start with, it’s a non-negligible amount of work to get that set up. I feel like the scenario in which this is useful is kinda narrow: when I have a very good idea of exactly what I want, but some step along the way is a hassle. General software engineering, like making a whole app, is far too open ended, and most of the sub-problems I encounter in software engineering seem either too open ended or too small to benefit from this approach.

            That also doesn’t account for the speed of models. My experience is a ~20GB locally hosted model takes like 1-5 minutes to produce a good length response, and the few times I have used online models they are often slower. A few minutes per iteration, accounting for debugging when it goes off track, is not exactly fast or hassle free.

            I just feel like the trade off where using AI vs doing it all myself is pretty limited in when AI offers an advantage.

    • dwemthy@lemmy.world
      link
      fedilink
      English
      arrow-up
      7
      ·
      12 days ago

      No matter how much I carefully structure a prompt, define specific behaviors in skills, and tweak the agent md files it will still just go do something I don’t tell it to or not do something it’s got really specific instructions for. We have to write all code by LLM now at work and I’m trying to do my due diligence to review code before putting it up for PR. 9 times out of 10 when I tell it to show me a diff before committing it silently runs git diff in the background and prints “that’s the full diff”. That’s with some basic “here’s what I want when I ask for a diff” in the base context.

      • peopleproblems@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        12 days ago

        Tbf I think its the context limit that makes things hard.

        1m token context is so stupidly low for all of the input we consider and filter in real time.

    • moopet@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      6
      ·
      11 days ago

      Yeah, every time I say this to someone they say it’s fine when they use it because they pay for the latest expensive model. Except they’ve been saying that for the last couple of years, so I guess they were lying back then…?

    • MeatPilot@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      4
      ·
      edit-2
      12 days ago

      Where I am forced to encounter advanced LLMs is every company rolled out replacements for my “dumb” assistants. Like Alexa on an echo dot or Gemini on Android auto.

      What I ask them to do does not take significant processing power. I need you to set a 5min timer. Not talk to me like you’re alive. Don’t overthink things, just do simple things.

      All that extra banter they added in eats up processing power. So now it’s slower and works like shit because it’s formulating some complex response to my request to beep in 5 minutes. Just set the timer, my hands are covered in raw chicken juice.

      • aesthelete@lemmy.world
        link
        fedilink
        English
        arrow-up
        3
        ·
        12 days ago

        You can disable Android auto’s Gemini by setting the digital assistant on your android phone to “none” under default apps.

        If you use these things more generally that might hurt you more than help, but it got rid of the bloviating idiot in the way of the old useful voice that gave me directions.

        • MeatPilot@sh.itjust.works
          link
          fedilink
          English
          arrow-up
          2
          ·
          11 days ago

          Thanks I’ll have to dig deeper. I figured out dumbing down Alexa but haven’t dug into Gemini yet.

          I just wish their was an alternative besides Apple and Google for car things. Sooner or later the choice of having it disabled is going to quietly go away.

          • aesthelete@lemmy.world
            link
            fedilink
            English
            arrow-up
            1
            ·
            edit-2
            11 days ago

            Turning off the assistant entirely isn’t a step I’d expect a lot of people to take (because they use the assistants). I think it’ll stick around if you can tolerate it, because it’s likely provided for certain parties that will be very irritated if that ability is removed.

            I not only can tolerate it, but it’s what I prefer. I wish they’d put “hold power button” back to show the restart dialog, I never used the assistant at all, and I even turned off that “google feed screen” that sits to the left of normal android screens. I love to disable junk.

    • teslasaur@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      ·
      12 days ago

      I use AI all the time to parse log files for errors. Or write up simple scripts to do things that are one-offs or test of concept. Most, if not all succeed. I made a parser that translates the config of one brand of switches to another one, worked perfectly.

      In what way are you using an LLM? It sounds like you’re asking it moral questions, to which it of course can’t give you an answer in any sort of objective sense.

      • ThirdConsul@lemmy.zip
        link
        fedilink
        English
        arrow-up
        1
        ·
        edit-2
        11 days ago

        parse log files for errors

        Surely you don’t tell your LLM “ingest the whole 10GB log file, look for errors”?

        made a parser that translates the config of one brand of switches to another one, worked perfectly

        I believe we are all saying in the thread that minor things do work correctly. Because a config map is not a hard thing to produce.

        • teslasaur@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          11 days ago

          Depends on the error obviously.

          Easy for you to say. There were none that existed, certainly not from the manufacturers. I may have been able to hobble it together in a day or two, but it took less than 30 min instead.

    • Snapz@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      ·
      11 days ago

      It’s a big club and you ain’t in it. They have to sell it until the end and this guy is on the outs and not getting invited to the “good” yacht parties at the moment.

      The judgement you are considering from this person is the same judgement (and morality) that thought it was fine to be a close and frequent associate of jeffrey epstein.

      Bill gates is a desperate, broken man. He is the illusion of a man you once admired vaguely and that is the only remaining currency he holds that matters to him. The best thing about bill is the wife who left him.

      • jj4211@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        ·
        11 days ago

        Problem is that Gates isn’t really “in the loop” and doesn’t have especially valuable insight.

        His position in tech was always a bit removed from the core technologist work, and now his exposure is a telephone game with people that are as distant from the tech as he was.

        It is really going around with no shortage of commentators spewing out guesswork, but Gates is given more credibility by virtue of his role 30 years ago.

      • treadful@lemmy.zip
        link
        fedilink
        English
        arrow-up
        1
        ·
        12 days ago

        That’s a fair point. I just don’t know if the future they see can become a reality.

        • redballooon@lemmy.world
          link
          fedilink
          English
          arrow-up
          2
          ·
          edit-2
          12 days ago

          I said that a year ago, and half a year ago, too, expecting the S curve to hit and the technology to flatten out at some upper limit. But instead we got the agent loop, and models that make really good use of it, the releases only get faster and faster, and real improvements with each one, either faster and cheaper, but just as good, or actually a good deal more capable. I’m seeing signs of behavior that is more than just a good auto complete, and think there is actual intelligence.

          Even should the S curve start to turn towards slowing down now ( and it doesn’t look like it), the ceiling that it’s going towards is so high, I am with Bill Gates here. We are not prepared for what is coming.

          • treadful@lemmy.zip
            link
            fedilink
            English
            arrow-up
            5
            ·
            12 days ago

            I’m seeing signs of behavior that is more than just a good auto complete, and think there is actual intelligence.

            I suggest you be real careful not to anthropomorphize these systems. To me this sounds almost like AI psychosis.

            • redballooon@lemmy.world
              link
              fedilink
              English
              arrow-up
              2
              ·
              12 days ago

              AI psychosis is what I accused by precious boss of, when he fabulated about replacing all his employees based on gpt-4o. What we have today is something completely different.

              Also, I’m not anthropomorphize these things. Intelligence is not a uniquely human quality.

            • protist@retrofed.com
              link
              fedilink
              English
              arrow-up
              2
              ·
              11 days ago

              I also don’t agree that they have any actual intelligence, but this is definitely not AI psychosis. AI psychosis is a pervasive pattern of delusional thinking and not just one thought you disagree with

              • treadful@lemmy.zip
                link
                fedilink
                English
                arrow-up
                1
                ·
                11 days ago

                I agree. That’s why I said “almost.” But AI psychosis starts with recognizing these things as sentient.

    • jerakor@startrek.website
      link
      fedilink
      English
      arrow-up
      3
      ·
      12 days ago

      The bar is not that AI needs to be right. It just needs to be more right than the employee willing to do the job for the price it costs.

      People have been bad at their jobs for years. Now computers can be to.

    • dropdrip@lemmy.ml
      link
      fedilink
      English
      arrow-up
      2
      ·
      edit-2
      11 days ago

      I’m hesitant to comment because this topic can become morose. I don’t think these commentators are talking about ChatGPT or whatever dribbles out into the consumer sphere. I think their concern is the extreme dis-balance of wealth and the ability for automated systems to exacerbate that. It’s not really about capital as capital is a form of control on labour; what if you removed capital entirely as you had a more effective mechanism of control? This isn’t about computers writing your assignments or ‘taking your job’, it’s a potential magnification of what technology innately does, but to an absurd degree. Technology allows for fewer men to control more. What happens when a small group of individuals control a nation’s economy?

      ‘AI’ was used to elect Trump; ‘AI’ is manipulating stocks; ‘AI’ is being used in propaganda; ‘AI’ is forcing economic rents to increase; etc…