• @sc_griffith
    link
    English
    266 months ago

    if I copy a coherent sentence into my clipboard, my clipboard becomes capable of consistently making coherent statements

    • Kogasa
      link
      fedilink
      English
      -4
      edit-2
      6 months ago

      Yes, but that’s not how LLMs work. My statement depends heavily on the fact that a LLM like GPT is coaxed into coherence by unsupervised or semi-supervised training. That the training process works is the evidence of an internal model (of language/related concepts), not just the fact that something outputs coherent statements.

      • @selfA
        link
        English
        146 months ago

        let me free up some of your time so you can go figure out how LLMs actually work

      • @sc_griffith
        link
        English
        14
        edit-2
        6 months ago

        if I have a bot pick a random book and copy the first sentence into my clipboard, my clipboard becomes capable of consistently making coherent statements. unsupervised training 👍

      • adderaline
        link
        fedilink
        English
        126 months ago

        this isn’t necessarily true. patterns in data aren’t by nature proof of an underlying system of logic. if you run the line-fitting machine on any kind of data, its going to output a line. considering just how much data is encoded into these transformers, i don’t think we can conclusively say that it has a underlying conception of how language works, much less an understanding of the concepts that language represents. it could really just be using the vast quantities of data it has to output approximately correct statements. there’s absolutely structure there, but it doesn’t have to have the kind of structured understanding humans have about language to produce language, in the same way a less sophisticated machine learning model doesn’t have to know what kind of data its fitting a line to to make a line.