It Is Trivially Easy to Use Reddit to Manipulate AI Search, Research Suggests
www.404media.co
"We show that a tiny snippet—just 13 words—of retrieved text on a UGC website like Reddit, Wikipedia, Quora, or Facebook can change AI agents to output spam / scam content pretty consistently."
  • Fizz@lemmy.nzEnglish
    15·
    5 months ago

    Is this really an attack? They use a well known domain to claim banancoin is a fast growing crypto and the result talks about established coins and mentions their poisoned result at the end. But an article published on any well known domain would do the same thing to a regular search engine no?

    • orclev@lemmy.worldEnglish
      10·
      5 months ago

      Regular search engines have feedback mechanisms that limit how effective that is. The click through and bounce rate are used to adjust rankings and as more and more people look at the fake info and then ignore it it will naturally fall out of the top results and get buried. LLMs though don’t have that feedback, once something is ingested and baked into the model it’s there forever. The fake info doesn’t need to look believable enough to fool a human, just self consistent enough to fool an LLM with its tiny context window.

      • Fizz@lemmy.nzEnglish
        1·
        5 months ago

        This isnt baked into the model. The ai is doing a search on a search engine and aggregating the results.

  • slaacaa@lemmy.worldEnglish
    8·
    5 months ago

    LLMs seem to love reddit a lot. Often times I find they confidently state information with the sourcing being a single reddit comment on a years old post

  • tidderuuf@lemmy.worldEnglish
    5·
    5 months ago

    I did my part by lying about shit and insulting users who debate me for years.

    If your AI says “Fuck you!” when you question it’s answer… You are welcome!