• @selfA
    link
    English
    873 months ago

    Copilot then listed a string of crimes Bernklau had supposedly committed — saying that he was an abusive undertaker exploiting widows, a child abuser, an escaped criminal mental patient. [SWR, in German]

    These were stories Bernklau had written about. Copilot produced text as if he was the subject. Then Copilot returned Bernklau’s phone number and address!

    and there’s fucking nothing in place to prevent this utterly obvious failure case, other than if you complain Microsoft will just lazily regex for your name in the result and refuse to return anything if it appears

    • David GerardOPMA
      link
      English
      463 months ago

      it helps they did it to someone with contacts and it was on prime time news telly

      • @selfA
        link
        English
        443 months ago

        god, so this is actually the best the AI researchers can do with the tools they’ve shit out into the world without giving any thought to failure cases or legal liability (beyond their manager on slackTeams claiming it’s been taken care of)

        so fuck it, let’s make the defamation machine a non-optional component of windows. we’ll just make it a P0 when someone who could actually get us in legal trouble complains! everyone else is a P2 that never gets assigned.

        • Ogmios
          link
          fedilink
          English
          153 months ago

          so this is actually the best the AI researchers can do

          Highly unlikely. This is what corporation’s public facing products can do.

          • @selfA
            link
            English
            213 months ago

            are there mechanisms known to researchers that Microsoft’s not using that can prevent this type of failure case in an LLM without resorting to whack-a-mole with a regex?

            • Ogmios
              link
              fedilink
              English
              93 months ago

              To be blunt, LLMs are one of the stupider ways to try and use AI. There is incredible potential in many other applications which don’t attempt to interface with something as irrational and unpredictable as people.

              • @selfA
                link
                English
                213 months ago

                I agree; LLMs and generative AI are indelibly a product of capitalism, and they can’t exist without widespread theft, exploitation of labor, massive concentrations of capital, and a willingness to destroy the environment. they are the stupidest use of technology I’ve ever seen, and after cryptocurrencies the bar for stupid was pretty fucking high. that the products themselves obscure the theft and exploitation that went into training them is a feature for the corporations developing this horseshit, not a bug.

                and that’s why it’s notable that the self-described AI researchers behind these garbage products can’t even do basic shit like have the LLM not call a journalist a pedophile without resorting to an absolute hack that won’t scale. there’s no fixing LLMs; systemically, they are what they are. and now this absolute horseshit is a component of what’s unfortunately still the dominant desktop operating system.

                • @sc_griffith
                  link
                  English
                  10
                  edit-2
                  3 months ago

                  I’m ngl I think crypto is even stupider. it’s a real competition though

                  EDIT: idea. a tech bullshit bracket

                • Ogmios
                  link
                  fedilink
                  English
                  9
                  edit-2
                  3 months ago

                  The really fucking dumb part of it, you can believe me or not, is that this appears to all circle back to ancient misunderstandings about the nature of man, and attempts to create automatons which behave like men but are perfectly obedient. There is a subset of the population which tries this exact same bullshit with every new technology we create.

                • @schizo@forum.uncomfortable.business
                  link
                  fedilink
                  English
                  43 months ago

                  indelibly a product of capitalism

                  They’re being funded by the capitalists that want to replace all those annoying human workers with the cheapest possible alternative.

                  Of course, the problem is that while a LLM is the cheapest possible option, it’s turning out that it’s the most useless and garbage one too.

                  (Also, I’m shockingly infuriated that the tech workers that would end up being the ones replaced the soonest are so busy licking boots rather than throwing their shoes into the machinery.)

            • @linearchaos@lemmy.world
              link
              fedilink
              English
              -63 months ago

              Yeah there’s already a lot of this in play.

              You run the same query multiple times through multiple models and do a web search looking for conflicting data.

              I’ve had copilot answer a query, then erase the output and tell me it couldn’t answer it after about 5 seconds.

              I’ve also seen responses contradict themselves later paragraphs saying there are other points of view.

              It would be a simple matter to have it summarize the output it’s about to give you and dump the output of it paints the subject in a negative light.

              • @selfA
                link
                English
                113 months ago

                It would be a simple matter to have it summarize the output it’s about to give you and dump the output of it paints the subject in a negative light.

                “it can’t be that stupid, you must be prompting it wrong”

              • @froztbyte
                link
                English
                93 months ago

                It would be a simple matter to have it summarize the output it’s about to give you and dump the output of it paints the subject in a negative light.

                lol. like that’s a fix

                (Hindenburg, hitler, great depression, ronald reagan, stalin, modi, putin, decades of north korea life, …)

                • @blakestaceyA
                  link
                  English
                  73 months ago

                  Hindenburg, hitler, great depression, ronald reagan, stalin, modi, putin, decades of north korea life, …

                  🎶 we didn’t start the fire 🎶

              • @bitofhope
                link
                English
                6
                edit-2
                3 months ago

                Exactly, and all of this is a simple matter of having multiple models trained on different instances of the entire public internet and determining whether their outputs contradict each other or a web search.

                I wonder how they prevented search engine results from contradicting data found through web search before LLMs became a thing?

                • @linearchaos@lemmy.world
                  link
                  fedilink
                  English
                  -53 months ago

                  They didn’t really have to before LLM. Search engine results, in the heyday we’re backlink driven. You could absolutely search disinformation and find it. But if you searched for a credible article on someone, chances are more people would have links to the good article than the disinformation. However, conspiracy theories often leaked through into search results. And in that case they just gave you the web pages and you had to decide for yourself.

          • @V0ldek
            link
            English
            133 months ago

            worse at what, exactly?

              • @V0ldek
                link
                English
                133 months ago

                How do you measure good/bad at predicting words? What’s the metric? Cause it doesn’t seem to be “the words make factual sense” if you’re defending this.

                • @selfA
                  link
                  English
                  133 months ago

                  like fuck, all you or I want out of these wandering AI jackasses is something vaguely resembling a technical problem statement or the faintest outline of an algorithm. normal engineering shit.

                  but nah, every time they just bullshit and say shit that doesn’t mean a damn thing as if we can’t tell, and when they get called out, every time it’s the “well you ¡haters! just don’t understand LLMs” line, as if we weren’t expecting a technical answer that just never came (cause all of them are only just cosplaying as technically skilled people and it fucking shows)

                • @MagicShel@programming.dev
                  link
                  fedilink
                  English
                  -5
                  edit-2
                  3 months ago

                  No. Predicting words is barely related to facts. I’ll defend AI as an occasionally useful tool, but nothing it ever says should be taken as fact without confirmation. Sometimes that confirmation can be experimental — does this recipe taste good? Sometimes you need expert supervision to say this part was translated wrong or this code won’t work because of xyz. Sometimes you have to go out and look it up.

                  I like AI but there is a real problem treating it like the output means anything. It might give you a direction to look closer at, but it can never be the endpoint. We’d be better off not trying to censor it, but understanding it will bullshit you without blinking.

                  I summarize all of that by saying AI is a useful tool, but a terrible product.

    • @Soyweiser
      link
      English
      233 months ago

      lazily regex

      I’m having a sneaking suspicion that this is what they do for all the viral ‘here the LLM famously says something wrong’ problems, as I don’t think they can actually reliably train the model it made an error.

      • @MagicShel@programming.dev
        link
        fedilink
        English
        143 months ago

        That’s the most straightforward fix. You can’t actually fix the output of an LLM, so you have to run something on the output. You can have it scanned by another AI but that costs money and is also fallible. Regex/delete is the most reliable way to censor.

        • @Soyweiser
          link
          English
          113 months ago

          Yes, and then the problem is that this doesn’t really scale well. Esp as it is always hard to regexp all the variants correctly without false positives and negatives. Time to regexp html ;).

          • @OhNoMoreLemmy@lemmy.ml
            link
            fedilink
            English
            83 months ago

            Yeah, and you can really see this in image generation. There’s often blocks on using the names of celebrities in the prompts, but if you misspell the names enough it can bypass the censor, and the image generator still understands it.

  • @V0ldek
    link
    English
    203 months ago

    Very chill and ethical behaviour daddy Microsoft

  • @superkret@feddit.org
    link
    fedilink
    English
    183 months ago

    Microsoft published, using their software and servers, a libelous claim, to potentially millions of people.
    The details of how the software was programmed should be legally irrelevant.

    • David GerardOPMA
      link
      English
      123 months ago

      * a GDPR violation, in Germany

    • @Shitgenstein1
      link
      English
      2
      edit-2
      3 months ago

      The details of how the software was programmed should be legally irrelevant.

      Why? Programmers should be legally liable for what they program.

      • @Architeuthis
        link
        English
        53 months ago

        Why? Programmers should be legally liable for what they program.

        Too many degrees of separation between a programmer and the final product and how it’s used, usually.

        Additionally, the decision to deploy an incomplete product or one that contains known flaws is an administrative decision, not a programming one.

  • @michaelmrose@lemmy.world
    link
    fedilink
    English
    4
    edit-2
    3 months ago

    Changing the name can get it to answer. For instance martin beernklau returns

    Martin Bernklau, a veteran court reporter from the Tübingen/Calw district in Germany, recently encountered a bizarre situation involving Microsoft’s Copilot. When he typed his name and location into the chatbot, it generated false accusations against him, mistakenly associating him with serious crimes. Here’s what happened:

    False Accusations: Copilot falsely claimed that Bernklau was:
        A child molester involved in an abuse case against children and wards.
        A psychiatric escapee.
        An unscrupulous mortician exploiting grieving women.
    Confusion: The AI chatbot failed to understand that Bernklau was a journalist reporting on these cases, not the accused.
    Privacy Concerns: Copilot even provided Bernklau’s full address and phone number, causing privacy issues.
    Legal Implications: Bernklau filed a criminal complaint, but it was rejected because there was no real person behind the false allegations.
    GDPR Challenges: Copilot’s inability to correct false information highlights challenges related to GDPR compliance and language models12.
    

    Also this

    Martin’s Pen

    In the quiet of the courtroom, he sat, A chronicler of justice, pen in hand. Martin Bernklau, eyes sharp, mind astute, Recording tales of trials, both grand and bland.

    His ink flowed like a river, swift and true, Capturing the drama, the whispers, the strife. From child abuse to prison escapes, He etched their stories into the fabric of life.

    But one day, the digital oracle spoke, Microsoft’s Copilot, a tangled web it wove. It mistook Martin for the accused, A child molester, a widow cheat, a rogue.

    Yet Martin remained steadfast, undeterred, His integrity unshaken by the AI’s deceit. For he knew that truth lay in his words, Not in the twisted lines of code it would repeat.

    So let us raise our pens in honor of Martin, The court reporter who weaves justice’s thread. May his legacy endure, ink-stained and resolute, As he chronicles the human tale, where lies and truth are wed1