I usually don’t even understand the lingo they use. “Open-weighted” is the most recent one, then it usually goes down to specific “models” that everybody is supposed to know about.

These are my thoughts (I will stick to the vague “it” for now, but of course therein lies another question: “and how does all this apply to various specialised AIs”):

  • Is it really feasible to run it 100% locally? I know there’s plenty of people with very powerful rigs indeed, but still. Or are 99% of these people really saying “it would, in theory, be possible to run that locally, therefore your concerns are invalid”?
  • If yes to the previous: the software doesn’t come from nowhere and ultimately still relies on gas-turbine-powered datacenters and stolen IP and stolen personal data, no?

If what I wrote above is true, what exactly are people arguing when they say it’s still possible to use LLMs ethically or true to FOSS philosophy, because … ???

edit

Thanks to all who answered.

I guess it’s my fault for asking several questions in one, but this thread has attracted exactly the type of people I’m writing about; several even used the term “open-weighted models” without explaining it.

Asking to get arguments explained, I got more arguments instead.

  • iceberg314@slrpnk.netEnglish
    10·
    1 month ago

    I think it’s at least way more ethical to use locally hosted open weight AI. Mostly because a lot of these models are usually fine tuned and distilled to run on a single GPU and that is the direction the technology needs to develop to be sustainable.

    Obviously for most things it’s way more ethical to just not use AI or LLMs.

    I think people should at least experiment with running local models on their own hardware. It really puts it into perspective how much compute they use. Like the power, speed, and performance ratios of different size LLM and how absurd anything involving generating video is. I have a pretty beefy gaming PC and it took it like 6 minutes to generate 10 seconds of 720p video. And then that time goes up exponentially past 10 seconds. Like 20 minutes for a 15 second video

    Running on a small model is basically like playing a modern full sized video game. When I measure my PC with a power meter

    • mojofrododojo@lemmy.worldEnglish
      2·
      1 month ago

      I think people should at least experiment with running local models on their own hardware.

      show me the one that’s built without theft and I might consider it.

  • flamingo_pinyata@sopuli.xyzEnglish
    9·
    1 month ago

    Let me just answer this part

    Is it really feasible to run it 100% locally? I know there’s plenty of people with very powerful rigs indeed, but still. Or are 99% of these people really saying “it would, in theory, be possible to run that locally, therefore your concerns are invalid”?

    Yes absolutely, and you don’t even need an extremely powerful machine. Basic text generation will work on a macbook pro. If the hardware prices were at normal levels, buying a machine for 2-3k USD/EUR and self-hosting powerful models would be feasible. Right now the same hardware will run you about 10k, that why I don’t think it’s practical at this moment.

    However training is where the real cost lies. It’s pretty much impossible to train a model from 0 on anything resembling a personal machine. First you need all the data, measuring in millions of terabytes. And then you need to train the model on all of that.

    On a side note - I once tried training a GPT2 replica on my laptop with about 3GB of random text. The estimated time was 3 months. That’s the level of requirements we’re talking about.

  • Pratai@piefed.caEnglish
    9·
    1 month ago

    There is no argument to use AI ethically. Only bullshit they tell themselves so as to circumvent responsibility.

  • Sunshine@piefed.caEnglish
    4·
    1 month ago

    When it doesn’t use external disinformation causing data centres and runs from home generated solar power.

  • leaky_shower_thought@feddit.nlEnglish
    2·
    1 month ago

    Is it really feasible to run it 100% locally? I know there’s plenty of people with very powerful rigs indeed, but still. Or are 99% of these people really saying “it would, in theory, be possible to run that locally, therefore your concerns are invalid”?

    There’s usually a lot of caveats they don’t mention on this. Like saying it** is feasible to run it** locally! (** it depends on your rig, memory and storage constraints, model licensing constraints blah-di-dah, blah-di-dee)

    Yes, there are super small and simple models so this is technically true.