I know the main cause is AI but why now, why was there not this demand at the same time as when GPU prices started to sky rocket?
Also I dont see why AI needs RAM like VRAM I get it but why RAM?
why AI needs RAM like VRAM I get it but why RAM?
They use the same silicon wafers and OpenAI just made a deal to secure 40% of the global supply of wafers from both SK Hynix and Samsung (2 of the three large providers) ostensibly for project Stargate server farms. It gets so much worse, they made both deals on the same day without advising the other company, and have not provisioned any way to actually use (make chips from) the wafers. It looks more like they’re just trying to keep them out of the hands of competitors. See here for the gory details.
From there the laws of supply and demand and panic buying by everyone else took over, micron (the third big one) dropped out of the consumer market because they’re gonna make bank in the server market as the only unencumbered company. Consumer general purpose computer customers are royally boned. This will flow through into the SSD market as well.
TLDR: Fuck AI in general and Fuck ‘OpenAI’ and Sam Altman in particular. If you pray, pray that this deal gets a legal injunction in South Korea, coz you know the US will just applaud this shitfuckery.
Geeze. What a cunt.
Also I dont see why AI needs RAM like VRAM I get it but why RAM?
VRAM is still RAM.
Micron and Samsung have been accused of colluding in addition to utilizing the South Korean government as a bit of a puppet in their price wars. Now micron has formally announced they’re exiting the consumer market. They say that this will affect crucial, but obvious many 3rd parties use Micron as well. I think the odds of this announcement coming as a shock to either SK Hynix or Samsung is zero and most likely they are dividing their interests to help each other avoid competition to lower the price hikes they’ve fought for through years of stubborn supply “shortages”
It’s also possible that Micron and Samsung are colluding against SK Hynix, since that has been the defacto leader for DDR5
Corporate investment in data centers for AI is creating a very low supply and a very high demand. Inventory for individual product parts are not static or set in stone. One part may go up, one may go down.
One other caveat is that many RAM makers aren’t increasing their yields because they fear an impending AI bubble pop.
I honestly can’t wait for the bubble to pop. There are some very niche use cases where AI can provide actual benefits but overall it’s been such a slopfest. And companies keep trying to force it into everything, even when it makes their product infinitely worse. So damn annoying.
They’re truly going all in on the burn it all down next week so we can be fat and happy today method, let’s see how it works out for them cotton
Supply (not enough) and Demand (astronomical).
That’s it.
GPU = Bitcoin farming. Prices rose during COVID because of the shipping safety that went in place, plus all the nerds farming for Bitcoin who wanted 4 gpus at once, and that’s also when the PS5 launched (not a PCIE GPU, but still contributed)
RAM = needed to process ai tasks faster. Now with huge factory size ai data centers, lots of order got lots and lots of ram. I’d expect this to be the norm every 10ish years when ram gets updated
It hasn’t been viable to mine Bitcoin on GPU for about a decade, it is all CPU. I mean you can still obviously do it, but it isn’t worth it. You would get better results mining something else and swapping.
Different coins have different best suited hardware. Some by design, some by lack of design.
I think when it was GPUs it was also crypto. Also, it’s harder for two things to be a bottleneck at the same time
Maybe the current squeeze is because some players decided to set up their farms at the same time. About that I don’t know much
As for the purely technical “why RAM” - when they’re running a model it has to fit in the memory. A model which weights 8GB on the drive requires at least 8GB of RAM (rest for the os etc). AFAIK there have been some papers how to get more out of smaller weight but in general we are at “more is better” part of development of that technology. And probably, because of the hype, having service throttled outside of any plan might be worse RN than getting worse results. So if I were setting up an AI farm and had the money, “the tendency is to push it as far as we can”
And free market is figuring out who is the highest bidder
