- cross-posted to:
- news@lemmy.linuxuserspace.show
- cross-posted to:
- news@lemmy.linuxuserspace.show
You must log in or # to comment.
“To enable the massive 256GB/s memory bandwidth that Ryzen AI Max delivers, the LPDDR5x is soldered,” writes Framework CEO Nirav Patel in a post about today’s announcements. “We spent months working with AMD to explore ways around this but ultimately determined that it wasn’t technically feasible to land modular memory at high throughput with the 256-bit memory bus. Because the memory is non-upgradeable, we’re being deliberate in making memory pricing more reasonable than you might find with other brands.”
😒🍎
Edit: to be clear, I was only trying to point out that “we’re being deliberate in making memory pricing more reasonable than you might find with other brands” is clearly targeting the Mac Mini, because Apple likes to price-gouge on RAM upgrades. (“Unamused face looking at Apple,” get it? Maybe I emoji’d wrong.) My comment is not meant to be an opinion about the soldered RAM.
To be fair it starts with 32GB of RAM, which should be enough for most people. I know it’s a bit ironic that Framework have a non-upgradeable part, but I can’t see myself buying a 128GB machine and hoping to raise it any time in the future.
If you really need an upgradeable machine you wouldn’t be buying a mini-PC anyways, seems like they’re trying to capture a different market entirely.
seems like they’re trying to capture a different market entirely.
Yes that’s the problem.
That they want to sell cheap ai research machines to use for workstation?
That’s a poor attempt to knowingly misrepresent my statement.
No, it is a question
The answer is that they’re abandoning their principles to pursue some other market segment.
Although I guess it could be said to be like Porsche and Lamborghini selling SUVs to support the development of their sports cars…
I don’t understand how that answers my question
My biggest gripe about non replaceable components is the chance that they’ll fail. I’ve had pretty much every component die on me at some point. If it’s replaceable it’s fine because you just get a new component, but if it isn’t you now have an expensive brick.
I will admit that I haven’t had anything fail recently like in the past, I have a feeling the capacitor plague of the early 2000s influenced my opinion on replaceable parts.
I also don’t fall in the category of people that need soldered components in order to meet their demands, I’m happy with raspberry pis and used business PCs.
You can get an MS-A1 barebones from minisforum right now for like 215 - BYO cpu, ddr5, and m2. But it’s got oculink on the back (the pcie dock is 100, but not mandatory if you’re not going to use it). I think it’s supposed to be on sale for another couple days.
According to the CEO in the LTT video about this thing it was a design choice made by AMD because otherwise they cannot get the ram speed they advertise.
Which is fine, but there was no obligation for Framework to use that chip either.
In the same video it’s pointed out that this product wouldn’t exist at all without the AMD chip. It’s literally built around it.
Suppose the counter is that the market is chock full of modular options to build a system without framework.
In the laptop space, it’s their unique hook in a market that is otherwise devoid of modularity. In the desktop space, even the mini itx space, framework doesn’t really need to be serving that modularity requirement since it is so well served already. It might make it so I’m likely to ignore it completely, but I’m not going to be super bothered when I have so many other options
deleted by creator
Yeah hugely disappointed by this tbh. They should have made a gaming capable steam machine in cooperation with valve instead :)
They still could; this seems aimed at the AI/ML research space TBH
This is an AI chip designed primarily for running AI workflows. The fact that it can game is secondary
Yeah exactly, its worthless… Even the big players already admit to the AI hype being over. This is the worst possible thing to launch for them, its like they have no idea who their customers are.
I mean, it’s not. You can do aí workflows with this wonderful chip.
If you wanna game, go buy nvidia
Well, more specifically: why didn’t they try to go for LPCAMM?
From what I understand, they did try, but AMD couldn’t get it to work because of signal integrity issues.
Because you’d get like half the memory bandwidth to a product where performance is most likely bandwidth limited. Signal integrity is a bitch.
I thought LPCAMM was designed specifically to address the bandwidth and connectivity issues that crop up around high-bandwidth + low-voltage RAM?
Would 256GB/s be too slow for large llms?
It runs on the gpu
Many LLM operations rely on fast memory and gpus seem to have that. Even though their memory is soldered and vbios is practically a black box that is tightly controlled. Nothing on a GPU is modular or repairable without soldering skills(and tools).
Huh?
Framework releasing a Mac Mini was certainly not on my bingo card for this year.
I wasn’t prepared. I’ve been eyeing a mini for a while and this thing kills it on value compared to what I would get in a similar price point.
What alternatives were you considering, and how does the product from Framework compare?
Mac mini and studio. The overall power comparison remains to be seen but cost to spec ratio I would have had to spend over 6k and couldn’t have 16tb of memory, frameworks was around 3200.
Ok, should I know who framework is? I’ve been a PC gamer since forever and I’ve never heard of this company.
They make repairable laptops.
repairable and upgradable*
I know it’s an absolutely banal nitpick, but I think it’s unfortunately a revelation in the current laptop market that ~90% of a laptop stays good for a really really long time, and the other 10% can be upgraded piecemeal as the need arises. Obviously this was never news to the Desktop world, but laptop manufacturers got away with claiming this was impossible for laptops in the name of efficiency and portability.
deleted by creator
Lmao the news about this desktop is strangling their website to the point of needing a 45 minute waiting list
They did announce three major products today.
Yeah that touchscreen tablet convertible machine is what has me psyched. I’m not the target for it, and already own a 16, but I could see that thing selling well. I honestly think they came out with the desktop because they just kinda felt they needed a desktop.
I have a 16 and a 13, I thought I’d give away the 13 when I got the 16 but I keep using the 13 as well cause of how portable it is. Lot nicer to lounge about with the 13 than the 16.
I might get the 12 to replace my 13 and use it for drawing practice and web browsing. Performance wise it’d be a downgrade from my 1280p but I don’t really need the performance.
Guilty. This thing came out at the perfect time and I was considering building my own or a Mac mini but this has 95% of what I’m looking for for less than a spec compromised Mac mini. So I preordered. And I kept hitting refresh lol.
The Lemmy Lick strikes again!
Not really sure who this is for. With soldered RAM is less upgradeable than a regular PC.
AI nerds maybe? Sure got a lot of RAM in there potentially attached to a GPU.
But how capable is that really when compared to a 5090 or similar?
Not really sure who this is for.
Second sentence in the linked article.
Soldered on ram and GPU. Strange for Framework.
Apparently AMD couldn’t make the signal integrity work out with socketed RAM. (source: LTT video with Framework CEO)
IMHO: Up until now, using soldered RAM was lazy and cheap bullshit. But I do think we are at the limit of what’s reasonable to do over socketed RAM. In high performance datacenter applications, socketed RAM is on it’s way out (see: MI300A, Grace-{Hopper,Blackwell},Xeon Max), with onboard memory gaining ground. I think we’ll see the same trend on consumer stuff as well. Requirements on memory bandwidth and latency are going up with recent trends like powerful integrated graphics and AI-slop, and socketed RAM simply won’t work.
It’s sad, but in a few generations I think only the lower end consumer CPUs will be possible to use with socketed RAM. I’m betting the high performance consumer CPUs will require not only soldered, but on-board RAM.
Finally, some Grace Hopper to make everyone happy: https://youtube.com/watch?v=gYqF6-h9Cvg
I definitely wouldn’t mind soldered RAM if there’s still an expansion socket. Solder in at least a reasonable minimum (16G?) and not the cheap stuff but memory that can actually use the signal integrity advantage, I may want more RAM but it’s fine if it’s a bit slower. You can leave out the DIMM slot but then have at least one PCIe x16 expansion slot. A free one, one in addition to the GPU slot. PCIe latency isn’t stellar but on the upside, expansion boards would come with their own memory controllers, and push come to shove you can configure the faster RAM as cache / the expansion RAM as swap.
Heck, throw the memory into the CPU package. It’s not like there’s ever a situation where you don’t need RAM.
All your RAM needs to be the same speed unless you want to open up a rabbit hole. All attempts at that thus far have kinda flopped. You can make very good use of such systems, but I’ve only seen it succeed with software specifically tailored for that use case (say databases or simulations).
The way I see it, RAM in the future will be on package and non-expandable. CXL might get some traction, but naah.
Couldn’t you just treat the socketed ram like another layer of memory effectively meaning that L1-3 are on the CPU “L4” would be soldered RAM and then L5 would be extra socketed RAM? Alternatively couldn’t you just treat it like really fast swap?
Wrote a longer reply to someone else, but briefly, yes, you are correct. Kinda.
Caches won’t help with bandwidth-bound compute (read: ”AI”) it the streamed dataset is significantly larger than the cache. A cache will only speed up repeated access to a limited set of data.
Using it as cache would reduce total capacity as cache implies coherence, and treating it as ordinary swap would mean copying to main memory before you access it which is silly when you can access it directly. That is you’d want to write a couple of lines of kernel code to use it effectively but it’s nowhere close to rocket science. Nowhere near as complicated as making proper use of NUMA architectures.
Could it work?
Yes, but it would require:
- A redesigned memory controller capable of tiering RAM (which would be more complex).
- OS-level support for dynamically assigning memory usage based on speed (Operating systems and applications assume all RAM operates at the same speed).
- Applications/libraries optimized to take advantage of this tiering.
Right now, the easiest solution for fast, high-bandwidth RAM is just to solder all of it.
The cache hierarchy has flopped? People aren’t using swap?
NUMA also hasn’t flopped, it’s just that most systems aren’t multi socket, or clusters. Different memory speeds connected to the same CPU is not ideal and you don’t build a system like that but among upgraded systems that’s not rare at all and software-wise worst thing that’ll happen is you get the lower memory speed. Which you’d get anyway if you only had socketed RAM.
Yeah, the cache hierarchy is behaving kinda wonky lately. Many AI workloads (and that’s what’s driving development lately) are constrained by bandwidth, and cache will only help you with a part of that. Cache will help with repeated access, not as much with streaming access to datasets much larger than the cache (i.e. many current AI models).
Intel already tried selling CPUs with both on-package HBM and slotted DDR-RAM. No one wanted it, as the performance gains of the expensive HBM evaporated completely as soon as you touched memory out-of-package. (Assuming workloads bound by memory bandwidth, which currently dominate the compute market)
To get good performance out of that, you may need to explicitly code the memory transfers to enable prefetch (preferably asynchronous) from the slower memory into the faster, á la classic GPU programming. YMMW.
I wasn’t really thinking of HPC but my next gaming rig, TBH. The OS can move often accessed pages into faster RAM just as it can move busy threads to faster cores, gaining you some fps a second or two after alt-tabbing back to the game after messing around with firefox. If it wasn’t for memory controllers generally driving channels all at the same speed that could already be a thing right now. It definitely already was a thing back in the days of swapping out to spinning platters.
Not sure about HBM in CPUs in general but with packaging advancement any in-package stuff is only going to become cheaper, HBM, pedestrian bandwidth, doesn’t matter.
The thing is, consumers didn’t push Nvidias stock sky high, AI did. Microsoft isn’t pushing anything sane to consumers, Microsoft is pushing AI. AMD, Intel, Nvidia and Qualcomm are all pushing AI to consumers. Additionally, on the graphics side of things, AMD is pushing APUs to consumers. They are all pushing things that require higher memory bandwidth.
Consumer will get ”trickle down silicon”, like it or not. Out of package memory will die. Maybe not with you next gaming rig, but maybe the one after that.
In systems where memory speed are mismatched, the system runs at the slowest module’s speed. So literally making the soldered, faster memory slower. Why even have soldered memory at that point?
I’d assume the soldered memory to have a dedicated memory controller. There’s also no hard requirement that a single controller can’t drive different channels at different speeds. The only hard requirement is that one channel needs to run at one speed.
…and the whole thing becomes completely irrelevant when we’re talking about PCIe expansion cards the memory controller doesn’t care.
Sound like a downgrade to me I rather have capability of adding more ram than having a soldered limited one doesn’t matter if it’s high performance. Especially for consumer stuff.
Looking at my actual PCs built in the last 25 years or so, I tend to buy a lot of good spec ram up front and never touch it again. My desktop from 2011 has 16GB and the one from 2018 has 32GB. With both now running Linux, it still feels like plenty.
When I go to build my next system, if I could get a motherboard with 64 or 128GB soldered to it, AND it was like double the speed, I might go for that choice.
We just need to keep competition alive in that space to avoid the dumb price gouging you get with phones and Macs and stuff.
Sounds like a load of bullshit to feed useful idiots.
Honestly I upgrade every few years and isually have to purchase a new mobo anyhow. I do think this could lead to less options for mobos though.
I get it but imagine the GPU style markup when all mobos have a set amount of RAM. You’ll have two identical boards except for $30 worth of memory with a price spread of $200+. Not fun.
I don’t think you are wrong, but I don’t think you go far enough. In a few generations, the only option for top performance will be a SoC. You’ll get to pick which SoC you want and what box you want to put it in.
the only option for top performance will be a SoC
System in a Package (SiP) at least. Might not be efficient to etch the logic and that much memory onto the same silicon die, as the latest and greatest TSMC node will likely be much more expensive per square mm than the cutting edge memory production node from Samsung or whatever foundry where the memory is being made.
But with advanced packaging going the way it’s been over the last decade or so, it’s going to be hard to compete with the latency/throughout of an in-package interposer. You can only do so much with the vias/pathways on a printed circuit board.
You are correct, I’m referring to on package. Need more coffee.
No, I don’t think you owe an apology. It’s a super common terminology almost to the point where I wouldn’t really even consider it outright wrong to describe it as a SoC. It’s just that the blurred distinction between a single chip and multiple chiplets packaged together are almost impossible for an outsider to tell without really getting into the published spec sheets for a product (and sometimes may not even be known then).
It’s just more technically precise to describe them as SiP, even if SoC functionally means something quite similar (and the language may evolve to the point where the terms are interchangeable in practice).
There’s even the next iteration already happening: Cerebras is maling wafer-scale chipa with integrated SRAM. If you want to have the highest memory-bandwith to your cpu core it has to lay exactly next to it ON the chip.
Ultimately RAM and processor will probably be indistinguishable with the human eye.
Ye the soldered ram is for sure making me doubt framework now.
Signal integrity is a real issue with dimm modules. It’s the same reason you don’t see modular VRAM on GPUs. If the ram needs to behave like VRAM, it needs to run at VRAM speeds.
Then don’t make it work like that. Desktop PCs are modular and Framework made a worse product in terms of modularity and repairability, the main sales of Framework. Just, like… wtf. This Framework product is cursed and shouldn’t exist.
Not strange at all.
They’re a business that makes its money off of selling hype to morons.
Just buy a ThinkPad, if you’re thinking about buying a Framework…
The Framework Desktop is powered by an AMD Ryzen AI Max processor, a Radeon 8060S integrated GPU, and between 32GB and 128GB of soldered-in RAM.
The CPU and GPU are one piece of silicon, and they’re soldered to the motherboard. The RAM is also soldered down and not upgradeable once you’ve bought it, setting it apart from nearly every other board Framework sells.
It’d raise an eyebrow if it was a laptop but it’s a freakin’ desktop. Fuck you framework.
insanely hostile response to something like this. they attempted to have these parts replaceable, AMD physically couldn’t do it. they’ve still made it as repairable as possible, and will without a doubt be more repairable than similar devices using this chipset. fucking relax, being reactionary without being informed is dumb.
will without a doubt be more repairable than similar devices
It’s a desktop, they’re repairable unless you solder something in.
using this chipset
They could’ve gone with any other chipset, making the whole thing irrelevant to begin with, but they couldn’t please the AI crowd that way.
By similar devices I obviously mean ITX PCs with similar chipsets. The average PC isn’t giving you 128GB of VRAM for $2k.
Local AI is a difficult thing to do right now, making a product to allow people to use AI without giving up their privacy is great.
We need to stop bending over backwards to lower our standards for the people making money off of us.
Have higher standards.
Don’t be a useful idiot.
You need to get a grip. Framework is a private company that has done very well by its customers and supporters so far. Stop being so fucking negative.
Oof, found the proud consumer.
They always get upset when you point out how they’re being taken advantage of.
They can never willingly take a bad deal, right? 😉
you seriously have issues. and you’re a dick on top of that.
you’re either a troll or just a shitty person.
Wow, calm down.
This is how you react when people say we should have higher standards? I guess you really are insecure about your purchasing habits.
The consumerism runs deep with you. Now you’re going to throw a tantrum trying to find ways to defend being taken for a ride 😎
Seriously that’s really disappointing. It really seems like investors decides that they needed to “diversify” their offering and they need something with AI now … Framework was on a good path imo but of course a repairable laptop only goes so far since people can repair it and don’t need to replace it every 2 years (or maybe just replace the motherboard) so if you want to grow you need to make more products …
Honestly this is exactly the product I was waiting for minisforum to make. I think this is actually a pretty solid move.
Same, I was ready to just buy the first chinese mini pc that had the 128G strix halo processor. This is way better as it will have actual support and likely be better made.
I agree. We need less soldered RAM designs. I thought repairability was something they appreciated.
Well tbh they still do have repairable laptops, even new ones and all that, and the “excuse” is that the only way to properly use that specific AMD CPU is with that specific RAM and the non.soldered bus wasn’t enough, but still… i’ll stick to old #ThinkPads, thank you.
This is one stupid product. It really goes against everything the framework brand has identified with.
Desktops are already that, though. In order for them to distinguish themselves in the industry, they can’t just offer another modular desktop PC. They can’t offer prebuilts, or gaming towers, or small form factor units, or pre-specced you-build kits. They can’t even offer low-cost micro-desktops. All of those markets are saturated.
But they can offer a cheap Mac Studio alternative. Nobody’s cracked that nut yet. And it remains to be seen if this will be it, but it certainly seems like it’s lined up to.
I’m not super well informed, but a socketable AMD nuc form factor machine would’ve been nice, single pcie, m.2 and 2 sodimm ram slots would’ve been good. Could’ve even given the option to route the pcie slot externally and offered an add on egpu case that’s actually worth a damn a la mega drive/sega cd.
It’s a straight up gimmick flanderizing the brand identity.
I feel like this is a big miss by framework. Maybe I just don’t understand because I already own a Velka 3 that i used happily for years and building small form factor with standard parts seems better than what this is offering. Better as in better performance, aesthetics, space optimization, upgradeability - SFF is not a cheap or easy way to build a computer.
The biggest constraint building in the sub-5 liter format is GPU compatibility because not many manufacturers even make boards in the <180mm length category. Also can’t go much higher than 150-200 watts because cooling is so difficult. There are still options though, i rocked a PNY 1660 super for a long time, and the current most powerful option is a 4060ti. Although upgrades are limited to what manufacturers occasionally produce, it is upgradeable, and it is truly desktop performance.
On the CPU side, you can physically put in whatever CPU you want. The only limitation is that the cooler, alpenfohn black ridge or noctua l9a/l9i, probably won’t have a good time cooling 100+ watts without aggressive undervolting and power limits. 65 watts TDP still gives you a ryzen 7 9700x.
Motherboards have the SFF tax but are high quality in general. Flex ATX PSUs were a bit harder to find 5 or 6 years ago but now the black 600W enhance ENP is readily available from Velkase’s website. Drives and memory are completely standard. m.2 fits with the motherboard, 2.5in SATA also fits in one of the corners. Normal low profile DDR5 is replaceable / upgradeable.
What framework is releasing is more like a laptop board in a ~4 liter case and I really don’t like that in order to upgrade any part of CPU, GPU or memory you have to replace the entire board because it’s soldered on APU and not socketed or discrete components. Framework’s enclosure hasn’t been designed to hold a motherboard+discrete GPU and the board doesn’t have a PCIe slot if you wanted to attach a card via riser in another case. It could be worse but I don’t see this as a good use of development resources.
I think the biggest limiting factor for your mini PC will always be the VRAM and any workload that enjoys that fast RAM speed. Really, I think this mini PC from framework is only sensible for certain workloads. It was poised as a mobile chip and certainly is majorly power efficient. On the other hand I don’t think it is for large scaling but more for testing at home or working at home on the cheap. It isn’t something I expected from framework though as I expected them to maintain modularity and the only modularity here is the little USB cards and the 3D printed front panel designs lol
Edit
Personally I am in that niche market of high RAM speed. Also, access to high VRAM for occasional LLM testing. Though it is an AMD and I don’t know if am comfortable switching from Nvidia for that workload just yet. Renting a GPU is just barely cheap enough.
Calling it a gaming PC feels misleading. It’s definitely geared more towards enterprise/AI workloads. If you want upgradeable just buy a regular framework. This desktop is interesting but niche and doesn’t seem like it’s for gamers.
I think it’s like Apple-Niche
Question about how shared VRAM works
So I need to specify in the BIOS the split, and then it’s dedicated at runtime, or can I allocate VRAM dynamically as needed by workload?
On macos you don’t really have to think about this, so wondering how this compares.
It’s typically dynamic
It will most likely be dynamic, with the option to statically set it.
Now, can we have a cool European company doing similar stuff? At the rate it’s going I can’t decide whether I shouldn’t buy American because I don’t want to support a fascist country or because I’m afraid the country might crumble so badly that I can’t count on getting service for my device.
I could envision MNT Research trying this in the future, but not for now.
Wow, that stuff looks awesome! Thanks for sharing 🙏
I don’t get the point. Framework laptops are interesting because they are modular but for desktop PCs that’s the default. And Framework’s PCs are less modular than a standard PC because the RAM is soldered
That makes no sense - that’s more like Apple then…
I don’t know if it’s the case, but modular IO on PC maybe nice.
Much like their laptops, I’m all for the idea, but what makes this desirable by those of us with no interest in AI?
I’m out of that loop though I get that AI is typically graphics processing heavy, can this be taken advantage of with other things like video rendering?
I just don’t know exactly what an AI CPU such as the Ryzen AI Max offers over a non-AI equivalent processor.
There is a massive push right now for energy efficient alternatives to nvidia GPUs for AI/ML. PLENTY of companies are dumping massive amounts of money on macs and rapidly learning the lesson the rest of us learned decades ago in terms of power and performance.
The reality is that this is going to be marketed for AI because it has an APU which, keeping it simple, is a CPU+GPU. And plenty of companies are going to rush to buy them for that and a very limited subset will have a good experience because they don’t have time sensitive operations.
But yeah, this is very much geared for light-moderate gaming, video rendering, and HTPCs. That is what APUs are actually good for. They make amazing workstations. I could also see this potentially being very useful for a small business/household local LLM for stuff like code generation and the like but… those small scale models don’t need anywhere near these resources.
As for framework being involved: Someone has kindly explained to me that even though you have to replace the entire mobo to increase the amount of memory, you can still customize your side panels at any moment so I guess that is fitting the mission statement.
For modularity: There’s also modular front I/O using the existing USB-C cards, and everything they installed uses standard connectors.
what makes this desirable by those of us with no interest in AI?
Juat maybe not all products need to be for everyone.
Sometimes it’s fine if a product fits your label of “Not for me”.I hate how power hungry the regular desktop platform is so having capable APUs like this that will use less power at full load than a comparable CPU+GPU combo at idle, is great, though it needs to become a lot more affordable.
Production costs are not low either, and AMD still needs to profit. AMD’s APUs are already very affordable.
Much like their laptops
Its nothing like their laptops, thats the issue :/ Soldered in stuff all around, nonstandard parts that make it useless for use as a standard PC or gaming console.
Sorry, I was stating that “much like their laptops, I like the idea of these desktops.” I was not trying to insinuate that they themselves are alike.
This is not really that interesting and kinda weird given the non-upgradability, but I guess it’s good for AI workloads. It’s just not that unique compared to their laptops.
I’d love a mid-tower case with swappable front panel I/O and modular bays for optical drives; would’ve been the perfect product for Framework to make IMO.
The mini’s are the latest new hotness for desktop computing. I’ve been running a dirt cheap $90US, mini for 2 years now. It fits extremely well on my desk, just tucked in under the monitor leaving plenty of room for all the other tasks I do daily.
Will it play the latest hot new video game? Nope. But it will run OnlyOffice, FreeCAD and FreeDoom just fine.
It’s just not that unique compared to their laptops.
This’ll be a good sell for the useful idiot crowd that has been conditioned to think gaming laptops are the devil.
At first I was skeptical during the announcement and then I saw the amount of ram and the rack. Imho it is not for enduser but for business. In fact we have workloads that would be perfectly fit that computer so why not?
its definitely a small business and homelab focused device. ill 100% be getting one for some local AI compute in my lab.
