Hey all you beautiful selfhosters,
What are your suggestions for frugally obtaining HDDs in the current economic climate? Specifically the EU (Netherlands).
I’m looking at second hand drives, but even those go for €100+ now, with bad sectors and all.
Can we organise a collective AI datacenter robbery and doll out some stolen drives? 😁
Everyone asks “where do we get more storage?” and not “do we need to hoard all of this?”
The answer is yes.
Seconded.
Because the answer to the second question is a very clear “yes”.
With how the internet is going, I don’t think we will be able to get content from it in 5 to 10 years. It will be completely locked down, so all we have on our drives will be it. Back to mailing DVDs!
I think a big fuel for these storage anxieties is the very real situation we’re in right now, where we’re watching the “forever Internet” erode and crumble before our eyes, and getting rug-pulled from every direction service-wise, and losing access to media we don’t have a hard copy of.
I do wish there were a better way to pool all this storage for a common library of preservation…I mean I guess Internet Archive is like that but they’re constantly under attack. All this is under heaps of legal “gray area” and obviously the media titans want to force a rental-only-own-nothing world.
Right now we kinda have to become a scattered group of amateur historians and librarians, to preserve our culture.
I don’t say this to be mean so please don’t take it this way, but I think this mentality is… privileged? If the free internet goes then so does society as we know it, and the obscure french film collections from the 80s isn’t gonna do anything for you in that new reality. The things that need to be prepped are plain text and take up no space at all, in the grand scheme of things. It is feasible to self-host a text-only version of the entirety of Wikipedia, but nobody here is talking about that. It’s as if people think there’s gonna be some middle-ground where the internet is totally shut down and somehow life goes on as normal. You’d think priorities would shift a little away from media consumption towards “oh shit how do I learn how to filter my water”.
It would sure suck if you were the only person on earth with that french film collection, but do any historians actually know how to reach you? Have you made this information available? If not, then your archive is not useful.
If the free internet goes then so does society as we know it, and the obscure french film collections from the 80s isn’t gonna do anything for you in that new reality. It’s as if people think there’s gonna be some middle-ground where the internet is totally shut down and somehow life goes on as normal. You’d think priorities would shift a little away from media consumption towards “oh shit how do I learn how to filter my water”.
This has been my thinking for the very longest while. If/When the internet goes ‘somewhere’, commerce screeches to a halt, globally. We’re at a point where there is no going back to pen and paper. When commerce screeches to a halt, no one is going to be gunning for your NAS drive filled with movies. They will be gunning for whatever life sustaining resources you have to make theirs. You think people are crazy now…we’ve yet to plumb the depths of crazy.
Well, okay, you’re talking “end of the free and open Internet” which, yeah, would be a pretty big and terrible deal, and I really hope people and institutions that are able, are archiving such important things! (I should see how big that Wikipedia archive is for kicks lol). But yeah, we’d have much bigger problems.
We gotta support archive.org and our libraries for this reason! I would hate to be without them.
I personally wasn’t talking about digital “prepping” so much as I was talking about motivations for hoarding data of things we’re interested in. Our media becoming lost media because wealthy interests don’t give a crap or, worse, decided to censor it.
Not everyone is just hoarding Sailor Moon episodes; people have tons of books and manuals too just because it interests them. Lots of people preserve video game ROMs as well, which has thankfully kept those works from disappearing entirely.
Moving that information would become significantly more difficult without a free and open Internet, but that’s a different “threat model” worth its own conversation, I think.
For what you’re talking about. I bet you’d be interested in someone like Marion Stokes , for instance, who “data hoarded” tons of recordings of television news, and that archive actually proved useful to historians way later.
Thanks for your thoughtful comment and perspective. :)
People treat deleting like some dirty word, but all good collections need to be organized and pruned.
What you don’t want to hold onto The Wrecking Crew for your descendants?
Don’t get emotional
The answer is 42.
yes
We wouldn’t be here if we hadn’t already answered the second question affirmatively.
In my country we have a website that resells “old” and used server hardware, including HDDs for reasonable prices. Although that has gone up a lot over the last year or so.
Maybe you have something like that in the Netherlands? I recently bought an 18TB drive for around €400.
Storage is just expensive these days. Just like RAM.
https://datablocks.dev/ is a Dutch company.
Oh well … they’re not getting cheaper.
Amazing! Thanks
Marty, we need to go back…to the future!
(Mind you, those top out at 320GB per cartridge)
Kidding aside, for deep storage…?
LTO-8 is 12TB native per cartridge. A used LTO can be as little as $300 USD with a 12TB cart $65ish. Ancient LTO-3 can be had for like…$5…and stores upto 800GB per tape.
So…if it’s deep storage you want…that’s one insane option.
OTOH if you’re looking for a Jellyfin streamer…they’re tape, so random access would suck bad. You’d genuinely be better off optical media at that point lol
Though if we’re time travelling… DVD shufflers (400+ DVDs) were a thing for a minute. You’d have to write bridge software because they’re HDMI and com port only…hmm. Checking quickly, they seem to go for $100 USD…is this even possible…? You’d be limited to 1 stream at a time, but I can see a way to share that across multiple TVs with HDMI splitter…hmm…how do I carry RF remote signal from each room back to main unit…oh, I don’t need to, could I make a web ui that controls the shuffler via a Pi to RS-232, that you access on your phone?..Shit…i could do this.
$300…I could do this. I could make a clock work Jellyfin server…
No, stop. This is a dangerous rabbit hole.
PS: shit - I just thought of two better options - and one of them is even semi sane (store video at 480-540p on DVD as mkv, use Nvidia shield to upscale on fly to 1080p). Back of envelope maths suggests this would be around 3000 movies.
I should not be online this late at night with easy access to credit card.
Use a hdmi to ip adapter to stream to multiple locations on a local network. Learned about them when looking at security cameras.
Stop enabling my drug addiction :P
LTO-8 is 12TB native per cartridge. A used LTO can be as little as $300 USD with a 12TB cart $65ish. Ancient LTO-3 can be had for like…$5…and stores upto 800GB per tape.
how do you find so cheap LTO drives?
how do I carry RF remote signal from each room back to main unit…oh, I don’t need to, could I make a web ui that controls the shuffler via a Pi to RS-232, that you access on your phone?..Shit…i could do this.
you could also do an
RFIR remote bridge with two minimal PisIve seen LTO for sale in person at the $300 USD mark…but I admit that’s a rare occurrence / fire sale. So that “as little as” is probably not fair. Sorry and retracted.
EBay shows some in the $1200-1500 USD range (and maybe closer to $3-4K brand new).
That now makes me feel stupid for walking past one at $300 (“tape drive? Who the fuck needs that ancient shit”) but I’m willing to bet that wasn’t a lto-8, in hindsight.
Even my LTO-3 claim is not as remembered; I can find a lto-3 for $60 USD here locally (not the $5 I jokingly claimed), with cartridges in the $15 USD range.
Not bad, but not 12TB per cartridge.
All of this to say; the DVD shuffler + pi intermediary (+ NVIDIA shield if needed) is probably the genuinely better version of this. Bizarrely.
Ebay, particularly GoHardDrives, or sometimes you’ll find new drives from random sellers.
I also check ServerPartDeals. Drives are pricy these days, don’t sneeze near your NAS.
Edit: I’m not sure if they ship internationally or not, however.
2/3 of the drives I received from GoHardDrive failed within 2 years and all I got was a pre-bubble refund.
Oof. 🙃
They do, but there may be better local alternatives, since the shipping can be quite high.
Honestly, I’ve been taking a chance on eBay. If the price is close to $10/TB and the drive is an enterprise drive that is listed as known to be working, with a good return policy, I take the risk. I just run tests as soon as I have it so far, all have been good (eight purchases).
Same. I use these drives in a mirrored setup or to hold data that’s replaceable. If they make it a few months without showing errors they might get entrusted to something more important. I’ll shell out for a new drive for my constantly in use drives, luckily I read the warning signs and bought a few 20tb HDDs when ram and SSDs started skyrocketing. I’m kicking myself for not grabbing a few backup m.2 nvme and 2.5" SSDs, because I’m already doing the hardrive shuffle in my mini PCs. I’m fine living life with networked rusty spinners, but I really really don’t want to go back to spinny boot/high throughput drives.
What sort of tests do you run?
I attach it to my unRAID server as an unassigned device and run a SMART test.
I still have some IOMEGA Zip drives. LOL Man, I remember when those seemed inexhaustible.
You know the fun part is you could just about use a 750 zip disk to steam video. Read speeds are about 7.5mb/s…enough for 1-2 simultaneous 480p Jellyfin streams.
Shit…everybody about RAID and here we are suggesting RAIT. No school like old school.
I still think the “DVD shuffler clockwork JF server with AI upscale” idea would be more fun to build tho, because as stupid as it sounds, the maths adds up. It would be gloriously cursed, but 3000+ hours of video is 3000 + hours of video.
I used to have a DVD duplicator. Picked it up at an auction many, many years back then turned it for twice what I had in it. Something similar to this:
Frugal and recycled. Using a mix of old disks in OMV8 with a mergerfs array, suprisingly they amounted to 20TB, so saved a few bob.
If a disk fails in Mergerfs you loose the data on that disk only and not the array as you would with a pure jbod array.
There is a remove disk utility in the OMV8 mergerfs plugin that allows the data from a failing disk to be copied back to the array, if enough space is available, retaining the data from the failing disk. The disk can then be physically removed. If the array is short on space, add a disk to expand the array before removing the failing disk.
I’ve only ever used it with a failing disk, not a failed disk, I would guess that a backup would be required in that case.
The same applies to mergerfs outside of OMV8, this was easier for me.
I use snapraid for parity on top (OMV7). Works nicely with mergerfs. If a drive fails it can be easily rebuilt. You can use older smaller drives to do split parity (I got 3x 8TB as array and 2x 4TB for parity)
I usually scavenge old drives from work. On one hand they’re a bit smaller than I’d like them to be, but on the other hand they’re free except from the minor work and documentation involved in ensuring that no company related data remain.
Serverpartdeals?
€356 excluding USA to EU shipping… Oof!
This is where I go. It’s ridiculously expensive right now, though. I just paid nearly 2k USD for three large capacity drives for our local UnRaid box.
Get lucky - just as this was starting I saw a failed disk notification on my NAS so I ordered a new drive just before they went up. Then I realized it was a stale notification for the drive I replaced a year ago so I have a spare should one fail.
trash bins 😅 sometimes you can get permission, other times not. 🤷 I’ve gotten permission to dig in this one trash bin, and it had a ton of decent 3tb hdds from server rack. pretty hard use and 1/3 of the ones I picked was overheating but the rest were still good!
what kind of place has those trash bins?
My work sells commercial tier drives that were used by customers in NVRs for $10/TB. It’s honestly a great perk of the job for a data hoarder like myself with a fully redundant 100TB of stuff (aka 200TB of drives). I also got a dope 16 bay server chassis with slides and a few other components from them. I fully intend to drop like 2 grand on HDDs if/when I move to a new company, assuming they don’t fire me for AI or something first.
I’ve had good success buying second hand on eBay, but I bet you could also do worse than getting used parts off Gumtree, look for anyone selling a broken or outdated computer - or in the free section - and spend some time going through a pile of ewaste, shucking all the drives, and then running tests on them.
Do you happen to live in zuid holland?
I have 2 unopened Seagate IronWolf ST4000VN006 that I’ll be happy selling.
No idea what a fair price or your budget would be. That is if you are interested in these drives to begin with.
I’m in Hilversum but new, unopened drives will be out of my price range. I’m only capable of sub €50 purchases at the moment
Ah ok, I’ve had these drives for a while now but ended up using SSDs instead so they just sat in my parts drawer reminding me of a bad purchase xd.
Crazy how they are like 3x the price now from when i bought them in 2023
