So I have been doing a lot of research since building my first attempt at an unRAID server last week. I come across one piece of information that I would like some clarification on if possible, excuse me in advance if I get anything wrong here as I am very new to this!

 

I read on the snapRAID website something about the way that parity works in different software RAID solutions. The information I came across suggested that with unRAID if one of your files becomes corrupt somehow you basically can't fix it with unRAID. So unRAID will tell you that your parity is incorrect but will just rebuild the parity causing the file to now become permanently corrupt? Or worse still will update the parity as normal as if a new file has been added or modified without your knowledge meaning you will not realise the file is corrupt?

 

Could someone clarify for me if this is the case? So if you only have one copy of your data protected only with parity you could potentially have a big problem (I know this is not advisable)

 

So snapRAID's argument is that using snapRAID because the parity is manually refreshed you can identify potential file corruption before updating the parity and use the parity to update the file to it's original state.

not sure what snapRaid claims, but Parity will not protect you from file corruption.

also I am not sure how would you use parity to restore a corrupted file.

unRaid parity correction dose not  cause cause file to become corrupted, it's already is,

all it does is fixes the parity to the current state of data status so if your drive is failed you can recover it. if the data on the drive is corrupted it will be recovered as such.

Parity protect you from Device failure not data corruption.

the only way to recover from data corruption is from a backup.

 

 

now there are some fileSystems out there (ZFS, BTRFS to name a few) that have a data correction algorithm in them that helps/protects the data with use of checksum and metadata.

 

but unraid does not employ this FS at the moment.

 

The only way to continuously validate your data is to run a MD5 checksum of a file/drive that you can store off UnRAID once the initial files have been written and periodically re-run a new MD5 checksum to compare against your original. This will let you know if a file has changed due to corruption or not. Then, as vl1969 mentioned, if you have a backup you can replace the corrupted file.

 

Ideally if you want to make sure you have perfect data you always need to maintain a backup copy that you can rely on to restore in the event something happens to your primary data set. Or, for critical things like pictures you should maintain multiple copies, including offsite (I sync mine to MS OneDrive).

 

 

I've been running unRAID for years and I've yet to have a parity error for no reason, ie a file that just got corrupted. One error was due to a bug in an older version of unRAID during a disk replacement and another time was errors due to a data drive acting up and returning bad data which I replaced and rebuilt.

 

File corruption is a very general term. Most often it occurs due to a program bug where a file is "corrupted" because of a bug. Only a backup can protect you from this type of issue. Or some PAR2 style scheme that can reconstruct files back to some known state. (Is that what snapRAID does?)

 

Another cause of corruption is a bad sector preventing reading a part of a file. UnRaid does protect you from this, and repairs the disk at the same time.

 

Bitrot is a theoretically possible corruption due to the magnetic material on the disk losing its magnetic setting over time. Know that modern drives include error detection and correction to compensate for such degradation. I think if RobJ were here he'd say having a disk return bad data is impossible, and we'd have a fun debate about that. But bitrot is not a great fear to me.

 

I really like unRaid's real time parity feature. My files are immediately protected as soon as the copy to the array is complete. That is very important to me.

 

Note that a second "parity" disk feature is being considered as a future feature. It would allow for a dual disk failure. But in addition it may be able to triangulate on data corruption if it happened. Again we're out in the land of infinitesimal possibility, and wel have to wait and see what we can do with dual parity once more details emerge.

 

If you are seriously looking at ways to protect yourself, look at the support threads. Monitor them for a month. See the real issues that real users have. And protect yourself from those. I have yet to see anyone reporting "Bitrot ate my homework".

Thanks for the replies chaps, I've done a bit more research on this myself and I've basically decided it's not big enough an issue / risk for me to really care about.

 

I think as someone has rightly pointed out, I've only suffered file corruption once in a media file and that was on an external drive that I had my suspicions of anyway.

 

I believe bjp999 snapRAID suggests that because it is a snapshot RAID file system if you compare the parity snapshot of your existing data before taking a new snapshot you can in theory rebuild the disk / folder as you would recover from a failed disk in any other parity based system. But because it's a snapshot from the last time the parity check was done, in theory if the file had become corrupt and different from the parity you could recover from the parity. Whether this actually works, or I've even understood it correctly is another matter all together!!

well just FYI, but I have seen posts that suggest that Tom is contemplating a file system change to BTRFS some time in the future.

who knows in a few years we might get unRaid Array running on BTRFS and/or even on BTRFS Raid blocks ( as i see it we might be able to build an array using a btrfs multydisk raid capability.)  this way you get the protection of unRaid parity

but also protection of BTRFS CoW and Raid setup.

 

who knows ;-)