Recently I had an issue on my server which is a bit "unexpected"...

 

A drive went bad--it would get "write" errors, but unraid wouldn't mark the drive as bad.  Using the smart utilities, the hd was saying it couldn't reallocat the blocks (or something to that effect.  I replaced the drive and resynced it and everything seemed good. 

 

As I randomly checked some (of the 45k) pictures I have stored on there, I noticed about 10% of them were corrupted.  (That is the color would be off or pixels would be shifted.)  This was quite puzzling.  Then the unraid server's samaba deamon would crash, and one time the kernel crashed...

 

To make a long story short, after many failed resierfsck's, it turns out my memory was bad.  I removed the bad memory and everything was fine.  This includes the corrupted pictures.

 

 

While I'm greatly relieved that everything worked out ok, my questions are:

* How do I know when files get corrupted?  (I can't look at each file every X days to make sure they are ok!)

* What's the best backup strategy? My concern is that I don't want to backup the corrupted version...so a "time machine" type backup may be the best one, but what works like that with unraid?

 

Thanks!

 

Ron

You might consider using TeraCopy to copy data to the array.  It reads the data back after the copy is finished and compares the the CRC  with the one calculated for the original source file.  This will assure the copy operation was successful.  Running a non-correcting parity check on whatever basis you consider necessary will assure that your data can still be successfully retrieved.

 

However, if you have a failure(s) on the non-correcting parity check, you must now proceed with caution!  As you have found, disk drives are NOT the only reason for a parity check failure!  Bad memory is one of the worst ones because it can make every other component look like it's bad.  You really have to carefully examine what is really happening and think about things.  EXAMPLE:  you have been weekly non-correcting parity checks for months and one week, you get a large numbers of errors.  Symptoms pointed to two simultaneous hard disk failures.  While this is possible, it is also not very probable.  Think things through three or four times so you have a clear path to working through the problem.  And start with the one which will be the least hazardous to your data if you are wrong.

When building an unraid server. part of the build process is to run memcheck to help prevent this sort of issue.

 

When migrating data to unaid from PC, I use a product called beyond compare in binary mode check. similar to terracopy, but it is a full binary compare. not just CRC.

When Migrating to unraid from a MAC, I use Rsync.

 

To back up an unraid itself.. you need a target to back up to.

 

some people use a second server (many times a second unraid server) or NAS and use Rsync.

Others backup to cloud based storage (Crashplan for example).

Some also mount a drive inside the unraid server that is not part of the array and rsync to that for some data (although massive hardware failure could kill the backup IMO). Some people do the same ting but to another share inside the protected array.

Some do 2 or 3 of the above options.