I store all of my files on my home files server using FreeNAS and using ZFS in RAIDZ2. (8 3TB drives). Built it back in 2007 and have only upgraded the drives since (750gb, 1TB, 2TB, 3TB)

Using ZFS all data is checksummed.

I backup the server to 3 Synology DS211+ NAS and 1 DS210 NAS using ext3 (synology doesn't support zfs). (2 3TB drives in each one)
I sync 1 NAS each week.

1 NAS for all data except media (RAID 0) 6TB (This one always stays offsite)
1 NAS for games (RAID1) 3TB
1 NAS for media (RAID0) 6TB
1 NAS for for things not media or games (RAID 1) 3TB

I store all NAS offsite at work which is about 5 minutes away.

Before using the NAS I used external enclosures but the enclosures I had would corrupt data with any disk configuration above 2TB. Even if they didn't you'd still have issues if your computer crashes, you disconnect the drive improperly,corruption issues if you use the ESATA interface, taking alot of time to chkdsk a GPT drive (and a ton of memory), and issues with compatibility with computers that don't support 3TB.

Next time I upgrade my file server I'll use a motherboard that supports ECC memory for extra protection.

I would like to sync all this data online but it's ALOT of data so no real way to do it (I couldn't imagine how long it would take to compare the difference between an offsite and onsite copy of data across the internet for 8+TB of data). So if there is some huge disaster in this area all my data will be lost (except if I can grab a NAS while running out the door or something)

There are tools you can run to check the hash of all of your files but I constantly update alot of my files so those tools would only really be usefull for data you don't change that often.

I guess if you had enough online storage space you could mabye create PAR files of all your local data and store them offsite. That way if you did have a local file corruption you could recover the data using the offsite par files. This wouldn't protect you from total drive failure just corruption of files though so probably only useful as another step in backups for ultimate protection.

I also use DropBox on my laptop and I store my financial and personal information in a truecrypt container and sync it there.

Leolo wrote:

I always try to buy quality components from reputable brands, but I cannot seem to avoid the dreaded problem of data corruption from time to time.

Leolo wrote:

Seagate Barracuda 1 TB SATA

There is your problem. I NEVER buy Seagate because they have such a terrible reputation. Go Samsung (They've just been bought by Seagate sadly so their old F3 and F4 drives are probably only worth going for) or Hitachi. Western Digital are a good choice too. 😀

are you using win7
try ntfs self healing
http://www.slickit.ca/2009/07/self-healing-ntfs.html

also you could try running the drive in a difernet mode
eg: normal insted of ahci
or compaable mode instead of enhanced mode

RichB93 wrote:

Leolo wrote:

I always try to buy quality components from reputable brands, but I cannot seem to avoid the dreaded problem of data corruption from time to time.

Leolo wrote:

Seagate Barracuda 1 TB SATA

There is your problem. I NEVER buy Seagate because they have such a terrible reputation. Go Samsung (They've just been bought by Seagate sadly so their old F3 and F4 drives are probably only worth going for) or Hitachi. Western Digital are a good choice too. 😀

Every maker of HDs has someone saying they make terrible hard drives 😉

yeah Dominus is right, Seagate is all I try to buy, I find their drives awesome... I tend to avoid WD though

Dominus wrote:

Every maker of HDs has someone saying they make terrible hard drives

Because every maker of hard drives occasionally releases a line of terrible hard drives. Which is exactly why die-hard brand loyalty to hard drive manufacturers is stupid. Seagate just happens to be the latest one to screw up, that's all...

Old Thrashbarg wrote:

Dominus wrote:

Every maker of HDs has someone saying they make terrible hard drives

Because every maker of hard drives occasionally releases a line of terrible hard drives. Which is exactly why die-hard brand loyalty to hard drive manufacturers is stupid. Seagate just happens to be the latest one to screw up, that's all...

The latest one? They've been doing it for YEARS now. The last good Seagate I had was a 40GB from 2005...

They certainly haven't been better since the whole .11 incident.

I never had data corruption on a main drive. I did have an odd USB drive with some duplicate data die on me though.

RichB93 wrote:

There is your problem. I NEVER buy Seagate because they have such a terrible reputation. Go Samsung (They've just been bought by Seagate sadly so their old F3 and F4 drives are probably only worth going for) or Hitachi. Western Digital are a good choice too. 😀

Not so sure about Hitachi (haven't really heard anything about them), but they did buy out IBM's hard drive division back in 2002 (aka the notorious crappy DeskStar or "Death Star" line). Also I've had a few WDs fail on me in the past, but those were from the late 90s/early 2000s. I've only owned one Maxtor drive and it's lasted me 6 years of heavy use so far, but *knock on wood.* It seems to be a gamble nowadays regardless of the manufacturer.

Regardless, I always keep a couple of spares. I don't trust anything that has mechanical components in it, or an IC.

Thanks a lot for your replies.

Wow, DosFreak, you sure have a thorough backup plan!! I've seen many middle sized companies (and some biggish ones) with backup strategies worse than yours!!

Is that for your personal information only, or are you also including your business data (if applicable) ??

I only have backups for my critical data, which are stored in a remote FTP server in another country here in Europe. (I pay a monthly fee to OVH, a european service provider).

But I also have lots of media downloaded from the internet (mostly abandonware, funny videos, movie trailers and random stuff) that I don't have backups of. It's data that won't make me go bankrupt if I lose it, but it's still annoying to lose it nonetheless.

I'll try with PAR files, seems to be a good option. Is there any recommended software for Windows that you know of?

Davros, yes, I'm using Windows 7, and I didn't know about those parameters. Thanks a lot for the tip!!

I'll enable the notifications. They don't seem to be enabled by default.

I guess the correct command lines would be:

fsutil repair set c: 0x09
fsutil repair set d: 0x09

Wouldn't they?

Tetrium, I've gotten corruption on a secondary drive. My main drive is now an intel SSD 320 series. Haven't had any problem until now, but it's still very new. Let's hope it's more reliable than the current generation of mechanical hard drives!!

which baracuda do you have? the 7200.11s were known to have tons of problems, but some where fixed using firmware updates
there are firmware updates also with the 7200.12s... you should try a firmware update first to see if that fixes your problem

and I believe some early nforce based motherboards had data corruption issues on the SATA controllers... firmware updates were released to counteract that issue

for me, I had no problems so far (knocks on wood), and I updated my firmware twice ever since I bought my drive last year... running the "CC49" firmware now, which not only improved speed, it actually allowed acoustic management to work properly unlike before which practically did nothing even when the BIOS setting was set to "enable"

I've not had such data corruption problems before too. Most of my HDDs are WD. Some are Seagate.

Sometimes defective IDE/sata cables and bad RAMs may mimic data corruption. But in these conditions, the errors may be random or as whole. Physical sector errors occur only while reading or writing at that particular sector alone, when the HDD itself is suspected.

Malik wrote:

Sometimes defective IDE/sata cables and bad RAMs may mimic data corruption.

This is very true. I noticed often the round IDE cables give me odd problems.

The only thing that worries me is if my offline database gets corrupted without me knowing it. But still, in the end theres not much you can do except making copies and/or using mirrored raid (though raid can't prevent corruption totally)

The only data corruption issue I had was on a Pentium 120 machine. I got this HP CD burner (4x I think) and all the Audio CDs had clicks and pops.

I simply couldn't figure it out and eventually tracked it down to a dodgy stick of memory. Took me AGES to track that one down. This was all when I was at school and quite the hardware noob 😁

Leolo wrote:

Wow, DosFreak, you sure have a thorough backup plan!! I've seen many middle sized companies (and some biggish ones) with backup strategies worse than yours!!

Is that for your personal information only, or are you also including your business data (if applicable) ??

Backups for everything. The media takes up the most space since I'm in the process of converting ALL of my DVD\Blu-ray to .mkv files......this is going to take awhile and will require alot of backups space. I notice a couple of years back that my discs had started to bit rot so I started to do backups but I quickly ran out of space and it takes way too much time but now I've kicked it back up again and am going through them slowly.

Streaming media still isn't as good as discs as far as video\sound quality\subtitles and extras so my converted DVD media will be better than that 😀 (Also I don't have to worry about my movies disapearring because of copyright issues, censored\cut scenes or my internet dropping)

All of this data is personal (although some of it is for work related purposes as well).

I'll try with PAR files, seems to be a good option. Is there any recommended software for Windows that you know of?

Do a google search for PAR2 there should be a couple of programs out there.

Pippy P. Poopypants wrote:

RichB93 wrote:

There is your problem. I NEVER buy Seagate because they have such a terrible reputation. Go Samsung (They've just been bought by Seagate sadly so their old F3 and F4 drives are probably only worth going for) or Hitachi. Western Digital are a good choice too. 😀

Not so sure about Hitachi (haven't really heard anything about them), but they did buy out IBM's hard drive division back in 2002 (aka the notorious crappy DeskStar or "Death Star" line). Also I've had a few WDs fail on me in the past, but those were from the late 90s/early 2000s. I've only owned one Maxtor drive and it's lasted me 6 years of heavy use so far, but *knock on wood.* It seems to be a gamble nowadays regardless of the manufacturer.

Regardless, I always keep a couple of spares. I don't trust anything that has mechanical components in it, or an IC.

Yeah, they did DeskStars for quite a while and they really improved them, amazing drives for the time. Were some of the first terabyte drives out there.

TheMAN wrote:

which baracuda do you have? the 7200.11s were known to have tons of problems, but some where fixed using firmware updates
there are firmware updates also with the 7200.12s... you should try a firmware update first to see if that fixes your problem

I have a Barracuda ST31000528AS, which I upgraded to version cc49 recently. But the problems have appeared a few weeks after that.

weird... I'm using the exact same HDD as you!