Running HDDs is what kills them, that's why RAID mirroring is NOT a backup solution, it's an availability solution.

HDDs have reached a price:size:density ratio where they can be a viable decent-value archival solution, with ever-increasing capacity, and instantly accessible.

1. Buy a handful of 2-3TB drives, try to get them from different manufacturing batches, or even better, different brands.
2. Run them all for a few weeks to weed out any DoA units, preferably with some stress test being run periodically.
3. Fill them.
4. Store them for 3-4 years, preferably somewhere offsite.
5. Buy a handful of whatever now sits in the price:capacity sweet spot.
6. Burn them in as per step 2
7. Transfer everything from old disks to new disks, adding in whatever you've accumulated since step 4.
8. Go to step 4.

ElectricMonk wrote:

Any idea what I should use as an archive medium, until a filesystem comes along to guard against bitrot? I *think* MS is coming out with one, but it's still in experimental/development phase.

I wouldn't trust any particular medium to protect against bitrot, I think the better ideal would be to have a system that can detect and correct it when it happens. Unfortunately, bitrot seems to be something nobody paid attention to until just recently, and it seems most filesystems and backup software still ignore it. I'd be interested in a backup utility that verifies checksums of unchanged files, for the purpose of identifying possible corruption, but I don't know if any have such a feature. I saw some reference to 'bacula' possibly being capable of this, but I didn't dig deep enough to figure it out.
I initially was only interested in SnapRAID because it gave some reasonably cost-effective redundancy for my less-than-critical video files. But it's ability to check for bitrot has made me interested in putting my critical files under it as well. Unfortunately it isn't intended for lots of small files, but I may use it that way regardless. (The critical stuff would still also get full backups)

If Microsoft is coming out with a filesystem that handles bitrot, I look forward to seeing that. I hope they make the feature optional though, because I wouldn't want it to slow anything down on a partition that isn't being used as a data archive.

Another opinion I've gathered on the web is that rare memory glitches may be a common cause of corruption. Some believe ECC memory is effective at reducing these events. I have no idea, but if you build a file server I'd consider ECC to be preferable.

shamino wrote:

If Microsoft is coming out with a filesystem that handles bitrot, I look forward to seeing that. I hope they make the feature optional though, because I wouldn't want it to slow anything down on a partition that isn't being used as a data archive.

Another opinion I've gathered on the web is that rare memory glitches may be a common cause of corruption. Some believe ECC memory is effective at reducing these events. I have no idea, but if you build a file server I'd consider ECC to be preferable.

IIRC, MS is developing ReFS (Resilient File System), but it's no where ready for prime time yet. I remember teaching NT4 Core tech classes, and MS made how NTFS "handles" bitrot an exam question, 🤣!

I'd considered ECC memory, but this particular PC won't ever be a file server. This is my gaming, GNS3, emulator rig. Once I get another job, and run GigE throughout the house (and multiple .11ac APs with controller), I intend on setting up one hoss of a VMware ESXi redundant server setup (with sharable nVidia GPUs, natch). If Mikrotik has finally gotten their crap together, I might use one of their Multilayer switches, unless I stick to the Cisco gear I already have.

I definitely need a large NAS/SAN, and to look into those M-Discs. I've about $250K worth of software I can't easily (or cheaply) replace. Plus, I hoard games. I have every game I've ever purchased since I was 10.

You'd mentioned wanting something "mad scientisty" - what about a robotic tape silo? Some of the older ones aren't terribly expensive, although capacity won't be outlandish (or you could go with an SL8500...because, you know, everyone needs a few petabytes of storage).

obobskivich wrote:

You'd mentioned wanting something "mad scientisty" - what about a robotic tape silo? Some of the older ones aren't terribly expensive, although capacity won't be outlandish (or you could go with an SL8500...because, you know, everyone needs a few petabytes of storage).

I've no idea what those are. I'll have to check those out.

Basically, since I want to get VMware certified (among all the others I'm getting and/or possess), I wanna go nuts with it. GigE everywhere, Wireless-HD to the multiple flatscreens (embedded in the wall, and touch screen ala Star Trek), and if I ever get remarried or have kids, everyone's computer except mine will be VMware clients. And I'd like a massive NAS/SAN partially for backup, partially for streaming HD movies to "thin" HTPCs in fafmily rooms.

Oh, and a buttload of MAME cabinets.

I was thinking about trying to get a cheap SGI ORIGIN 3000, or an RS-6000, until I saw the prices (and that the IBM box require a freaking muffler!) I do love the fact that IBM service manuals have one specific manual, that tells you how to look things up in the other bajillion manuals. (looking at you, AS/400).

I want to put 10-gig ethernet in my house. Sadly the price is still a bit high. But in a year or so, ig should drop closer to "affordable".

luckybob wrote:

I want to put 10-gig ethernet in my house. Sadly the price is still a bit high. But in a year or so, ig should drop closer to "affordable".

Keep an eye on RouterBoards from Mikrotik. I've only used their APs (Hautespot outdoor branded w/ Ubiquiti cards we swapped out), but they've got some very affordable gear, and they seem to be getting better all the time. Even the goons at SA are impressed, and deployments are increasing. I really liked their APs (the CLI and Gui were wonky, but that's cuz they're Latvian), but they're offering multilayer gigabit switches with built in .11b/g/n for like $600. And they can do almost everything the big boys can do, at almost wirespeed. You'll have to sell a kidney, and some limbs to get something like that from Cisco, Juniper. or Adtran.

BTW, Won't you need either CAT7, or use fiber runs with SFP+s to get 10GigE? I'm use to only seeing 10 and 40GigE on line cards, and those used short fiber cable with attached SFPs.

ElectricMonk wrote:

I've no idea what those are. I'll have to check those out.

Like this, on the small side:
http://www.ebay.com/itm/Compaq-StorageWorks-T … y-/291181406038

obobskivich wrote:

ElectricMonk wrote:

I've no idea what those are. I'll have to check those out.

Like this, on the small side:
http://www.ebay.com/itm/Compaq-StorageWorks-T … y-/291181406038

I must admit, I'm intrigued, and that retro styling kinda works in its favor.

Don't get me wrong - it would be outlandishly bulky and power hungry for the storage it offers, but...its a ROBOT. I figured that lets it classify as "mad scientist" style. The tapes aren't terribly expensive unless you're going after all 100 of them.

ElectricMonk wrote:

I've no idea what those are. I'll have to check those out.

Ever see the movie "Hackers"? That's with videotapes, but same idea. 😁

"MESS WITH THE BEST... DIE LIKE THE REST."

Stiletto wrote:

Ever see the movie "Hackers"? That's with videotapes, but same idea. 😁

"MESS WITH THE BEST... DIE LIKE THE REST."

That scene is actually what made me think of the tape robot... 🤣

http://www.youtube.com/watch?v=2efhrCxI4J0

obobskivich wrote:

That scene is actually what made me think of the tape robot... 🤣

http://www.youtube.com/watch?v=2efhrCxI4J0

There aren't words for how much I love/ironically love this movie. 🤣