unraidfan Posted July 25, 2021 Share Posted July 25, 2021 While doing a preclear of my drives for my setup, I thought I'd do a little reading about the safety precautions for bit-rot that Unraid might have in place. I was horrified to see that there is no such functionality to use the (2) Parity drive(s) to scrub and repair silent corruption. I have seen debates about ECC memory being the source (which I do not believe based on discussions I've had with engineers who worked at Sun and Apple who worked extensively in this space with ZFS in the early days and based on their evaluations where they lined up a lot of other scenarios that simply demonstrate a big "NO" to that thought process - bit-rot unequivocally exists and happens), etc. but in the end the result is that Unraid simply doesn't appear to facilitate silent corruption detection + repair. Even tools like DYNAMIX FILE INTEGRITY PLUGIN doesn't solve the problem since it doesn't allow you to recover from a corrupted file. What if your backup(s) has captured a corrupted snapshot and you simply no longer have an uncorrupted file anymore? I suppose BTRFS could be used on an individual disk basis, but again, how do you recover things quickly enough to avoid all bad scenarios? I don't see any automated solution there, but that would be nice if it exists. I think if it does exist this might be useful and I would definitely use BTRFS instead of XFS for this sanity saver. To state this even more clearly, I would LOVE to see ZFS be used in conjunction with Unraid to facilitate this perpetual scrubbing and recovery! 💙 But alas, it doesn't appear that this exists...... Anyone have a solution? I'm willing to buy an additional drive or two to facilitate perpetual scrubbing recovery if I could just use Unraid due to it's expandability attribute, but at this rate I just cannot tolerate the risk of silent corruption and will have to go with a ZFS based solution... 😿 Quote Link to comment
JorgeB Posted July 25, 2021 Share Posted July 25, 2021 With btrfs you can detect corruption, but since Unraid uses a separate filesystem for each array data device it can never be automatically fixed, you'd need to restore from backups, if/when Unraid starts allowing zfs it will be the same, both will be able to check and restore redundant pools though. Quote Link to comment
DivideBy0 Posted July 25, 2021 Share Posted July 25, 2021 (edited) 5 hours ago, JorgeB said: With btrfs you can detect corruption, but since Unraid uses a separate filesystem for each array data device it can never be automatically fixed, you'd need to restore from backups, if/when Unraid starts allowing zfs it will be the same, both will be able to check and restore redundant pools though. When is unraid planning to move to zfs? Is that on the roadmap? Edited July 25, 2021 by johnwhicker Quote Link to comment
unraidfan Posted July 25, 2021 Author Share Posted July 25, 2021 With ZFS one could do the (perhaps dreaded) `copies=2` which stores two copies. Definitely suboptimal, but something has to be done otherwise I don't see Unraid as being tenable now 😭 This makes me very very sad after spending an inordinate amount of time to get it to work on the servers I have. Quote Link to comment
JorgeB Posted July 26, 2021 Share Posted July 26, 2021 17 hours ago, johnwhicker said: When is unraid planning to move to zfs? Is that on the roadmap? It's been mentioned multiple times, but no ETA. 13 hours ago, unraidfan said: With ZFS one could do the (perhaps dreaded) `copies=2` which stores two copies. You could do the same with btrfs, but that is far from ideal, much better to just have backups since you can need them to recover from other situations anyway. Quote Link to comment
PeteAron Posted July 26, 2021 Share Posted July 26, 2021 If you'll forgive me, your post sounds like an open threat that developers must provide xyz or you will buy something else. If that's not what you're trying to say then you might reword it... if this is what you meant, well, best of luck to you mate. It seems to me that a second server, providing backup, is a solution. There must be hundreds of ways to use a backup server effectively. 1 Quote Link to comment
unraidfan Posted July 27, 2021 Author Share Posted July 27, 2021 I don't understand you @PeteAron, are you talking to me? Quote Link to comment
primeval_god Posted July 27, 2021 Share Posted July 27, 2021 On 7/24/2021 at 8:25 PM, unraidfan said: Even tools like DYNAMIX FILE INTEGRITY PLUGIN doesn't solve the problem since it doesn't allow you to recover from a corrupted file. What if your backup(s) has captured a corrupted snapshot and you simply no longer have an uncorrupted file anymore? The idea with the DYNAMIX FILE INTEGRITY PLUGIN is that you would run a check on a schedule often enough to detect issues before good copies are rotated out of your backup. Quote Link to comment
unraidfan Posted July 27, 2021 Author Share Posted July 27, 2021 (edited) Right @primeval_god but apparently SnapRAID is a more refined process and includes it in the parity storage from what I'm hearing? Edited July 27, 2021 by unraidfan Quote Link to comment
aarontry Posted February 26, 2022 Share Posted February 26, 2022 On 7/26/2021 at 7:34 PM, PeteAron said: It seems to me that a second server, providing backup, is a solution. This simply is not true. You could be backing up corrupted files. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.