jimmysticky Posted January 20, 2016 Share Posted January 20, 2016 Apparently Wiki defines bit-rot as something else. But talking about data-rot. How do you deal with that within unRAID? I have a similar forum going on in the FreeNAS forums. It seems that is what ZFS is all about. I'm just not sure I have the time to learn the complexities of freeNAS or the $ to burn on new hardware that it requires. On another thread, the use of the Btrfs file system was used with unRAID, but it was mentioned that the stability is questionable. I know that users here must be concerned about data rot and was wondering how you deal with it. Thanks! Link to comment
Chris Pollard Posted January 20, 2016 Share Posted January 20, 2016 I use file checksums and check them periodically. Doesn't help if I get corruption but at least I'll know its happening and can fix. I've got a Seagate Momentus XT in one of my servers as a cache drive which was silently corrupting files..... firmware fix, but was painful to figure out even when I knew files were getting corrupted. Link to comment
John_M Posted January 20, 2016 Share Posted January 20, 2016 I use this but I've been getting false positives reporting corruption of Mac OS X specific hidden files. I think it works well for general files and I'm testing a new version that excludes the Mac files, and I recommend it if it fits your needs. Note that it detects file rot but doesn't repair it. Link to comment
jimmysticky Posted January 20, 2016 Author Share Posted January 20, 2016 Thanks guys. I'm struggling with the notion that there is not a lot that can be done to repair rotten data with unRAID. Link to comment
JonathanM Posted January 20, 2016 Share Posted January 20, 2016 Thanks guys. I'm struggling with the notion that there is not a lot that can be done to repair rotten data with unRAID. If the data is important to you, it needs to be backed up. Period. If corruption is detected, restore from backup. There is no magic bullet that will recreate data without some form of backup. If you are relying on a single machine to keep your data safe from harm, you will get burned at some point. Doesn't matter which OS or what hardware you are running. That said, silent corruption is pretty rare. Accidental deletion or overwriting are MUCH more common. Backup, backup, backup. Link to comment
JorgeB Posted January 20, 2016 Share Posted January 20, 2016 There is no magic bullet that will recreate data without some form of backup. Completely agree, I separate my data in 3 categories: 1 - Most important and irreplaceable data - checksums and full backup on another unraid server 2 - Difficult to replace but can live with losing some stuff - checksums + 5% par2 3 - Easily replaceable data - checksums only Link to comment
jimmysticky Posted January 20, 2016 Author Share Posted January 20, 2016 Yes, I agree with the backups. Half my storage is used for backup. My array is 15TB on my pc. JBOD span. I have an enclosure with 15TB as the backup. And I have a 15TB off-site backup too. In regards to datarot, if one of my files on the array does incur a silent error, won't that just be passed onto the backup as well? My understanding (and I'm still learning) of data rot is that it is an error that the HDD does not know about. Therefore it assumes that the error is normal so it goes undetected. Link to comment
gundamguy Posted January 20, 2016 Share Posted January 20, 2016 Yes, I agree with the backups. Half my storage is used for backup. My array is 15TB on my pc. JBOD span. I have an enclosure with 15TB as the backup. And I have a 15TB off-site backup too. In regards to datarot, if one of my files on the array does incur a silent error, won't that just be passed onto the backup as well? My understanding (and I'm still learning) of data rot is that it is an error that the HDD does not know about. Therefore it assumes that the error is normal so it goes undetected. If you do it incorrectly yes... If you do it correctly no... It depends on if you are talking about an active backup (automatically backup at set intervals) yes that rot might be copied before you realize you've got a problem... If you mean a "Cold Storage" backup (one that isn't automatic) then you can check against good Checksums to make sure the backup is not corrupted. If you have an active but incremental backup you might avoid the corruption.... The details of what you do and how exactly you do it really matter in this case... Link to comment
jimmysticky Posted January 20, 2016 Author Share Posted January 20, 2016 Interesting. For me, I run the program freefilesync every week or so. It does a one-way mirror sync by looking at the data on my array and then it edits the data on the enclosure to mirror the array. I guess in my case if I have an error, it's going to send it to the backup EDIT: I should note, I'm not using unRAID. I'm still in the research phase. Link to comment
trurl Posted January 20, 2016 Share Posted January 20, 2016 Interesting. For me, I run the program freefilesync every week or so. It does a one-way mirror sync by looking at the data on my array and then it edits the data on the enclosure to mirror the array. I guess in my case if I have an error, it's going to send it to the backup EDIT: I should note, I'm not using unRAID. I'm still in the research phase. Not familiar with your software, but even that would depend on how it determines there are changes to be copied. For example, if the software looks at file timestamps, bitrot would not typically modify the timestamp of a file since it would not be the result of the OS writing the file. Link to comment
Recommended Posts
Archived
This topic is now archived and is closed to further replies.