• @T156@lemmy.world
    link
    fedilink
    English
    4016 days ago

    How did they estimate whether an LLM was used to write the text or not? Did they do it by hand, or using a detector?

    Since detectors are notorious for picking up ESL writers, or professionally written text as AI-Generated.

    • @sober_monk@lemmy.world
      link
      fedilink
      English
      23
      edit-2
      16 days ago

      They developed their own detector described in another paper. Basically, this reverse-engineers texts based on their vocabulary to provide an estimate on how much of them were ChatGPT.

    • Bob Robertson IX
      link
      fedilink
      English
      1816 days ago

      They just asked a few people if they thought it was written by an LLM. /s

      I mean, you can tell when something is written from ChatGPT, especially if the person isn’t using it for editing, but is just asking it to write a complaint or request. It is likely they are only counting the most obvious, so the actual count is higher.

    • @hypna@lemmy.world
      link
      fedilink
      English
      1116 days ago

      I don’t know of any reason that the proportion of ESL writers would have started trending up in 2022.