rick9814911 Posted September 30, 2017 Share Posted September 30, 2017 Hi, I recently started my unRAID trial and setup my array using BTRFS. From what I've read it should be stable enough, and I do make backups of course! One of the big reasons I went with BTRFS over XFS is to protect against bitrot. However, I was talking with a user on Reddit that said the following: "BTRFS can identify errors in a RAID0 or 'no raid' scenario, but only RAID1 can correct. (and 5 and 6, but they're not ready)" Is this true? Does BTRFS protect against bitrot when used on the array? Or does it only detect the errors but isn't able to correct them? Sorry if this has been documented before! I searched the forums but didn't find an answer to this. Any input would be much appreciated! Without the ability to fix bitrot errors I would probably choose XFS over BTRFS since it's been around much longer and seems to be very stable. 1 Quote Link to comment
JorgeB Posted September 30, 2017 Share Posted September 30, 2017 15 minutes ago, rick9814911 said: Or does it only detect the errors but isn't able to correct them? Correct, since each data disk is a separate filesystem and using the default profile, btrfs will detect a checksum error but won't be able to fix it, that is what backups are for. You could use dup profile for one or more disks, so data would be duplicated and any checksum error fixable, but obviously you'd lose half capacity on those disks. Quote Link to comment
rick9814911 Posted September 30, 2017 Author Share Posted September 30, 2017 (edited) 9 minutes ago, johnnie.black said: Correct, since each data disk is a separate filesystem and using the default profile, btrfs will detect a checksum error but won't be able to fix it, that is what backups are for. You could use dup profile for one or more disks, so data would be duplicated and any checksum error fixable, but obviously you'd lose half capacity on those disks. Thank you for clarifying! My only concern is that if I backup a file not knowing it's been corrupted, my backup will also be corrupt. That is the case right? Is there a recommended way to protect against bitrot on unRAID? I really love unRAID, but bitrot does worry me. Edited September 30, 2017 by rick9814911 Quote Link to comment
JorgeB Posted September 30, 2017 Share Posted September 30, 2017 Just now, rick9814911 said: My only concern is that if I backup a file not knowing it's been corrupted, my backup will also be corrupt. That is the case right? No, btrfs will error if it finds a checksum error, i.e., you won't be able to copy or play that file, you just need to check the log and will find a checksum error. Quote Link to comment
rick9814911 Posted September 30, 2017 Author Share Posted September 30, 2017 1 minute ago, johnnie.black said: No, btrfs will error if it finds a checksum error, i.e., you won't be able to copy or play that file, you just need to check the log and will find a checksum error. Excellent!! That's great news. Will it tell you which file is corrupt so I can restore it from the previous backup? Quote Link to comment
JorgeB Posted September 30, 2017 Share Posted September 30, 2017 4 minutes ago, rick9814911 said: Will it tell you which file is corrupt so I can restore it from the previous backup? Yes, file name will appear on the syslog, any time you want you can also run a scrub to check that everything is OK. Quote Link to comment
rick9814911 Posted September 30, 2017 Author Share Posted September 30, 2017 1 minute ago, johnnie.black said: Yes, file name will appear on the syslog, any time you want you can also run a scrub to check that everything is OK. Thank you so much! You've been a huge help. Quote Link to comment
ashman70 Posted September 30, 2017 Share Posted September 30, 2017 So would a scrub not potentially fix the problem? Or is it not the sort of problem that is fixable? Is the only choice restoring the corrupted file? Quote Link to comment
JorgeB Posted September 30, 2017 Share Posted September 30, 2017 9 minutes ago, ashman70 said: So would a scrub not potentially fix the problem? A scrub can only fix checksum errors on a redundant btrfs filesystem, e.g. a raid1 cache pool. Quote Link to comment
ashman70 Posted September 30, 2017 Share Posted September 30, 2017 On a speculative note, how common would you say Bitrot is? People talk about it but I've never heard of anyone experiencing it, which of course doesn't mean anything. Quote Link to comment
JorgeB Posted September 30, 2017 Share Posted September 30, 2017 4 minutes ago, ashman70 said: On a speculative note, how common would you say Bitrot is? Never seen it and I would guess no very common, but I did have a couple of situations where checksums were very valuable, and that's why I want them, once when there were read errors on a second disk during a rebuild of a failed disk (this was before dual parity) and another time when a disk red-baled during a disk to disk move, checksums allowed me to quickly find the affect files and replaced them from backups. 1 Quote Link to comment
johner Posted March 29, 2020 Share Posted March 29, 2020 so if Btrfs is used in an unraid array (with parity), when Btrfs raises the io error due to bitrot (if it does), wouldn't "unraid" then fix the file? isn't that the point of the unraid layer? I totally understand Btrfs can't in this scenario. given Btrfs redundancy. Quote Link to comment
JorgeB Posted March 29, 2020 Share Posted March 29, 2020 7 hours ago, johner said: wouldn't "unraid" then fix the file? isn't that the point of the unraid layer? If you knew the problem was only on the actual disk and parity was still correct you could rebuild the disk to correct the error. Quote Link to comment
johner Posted March 30, 2020 Share Posted March 30, 2020 I did a bit of reading, so it looks like the unraid layer is below the filesystem (files view) layer, it works on the blocks. therefore, there would need to be a way for unraid to ask Btrfs what blocks the bad file resides on and then recover them. I wonder if someone could take the open source code of unraid and create a new ZFS 'unraid' config 🙂 1 Quote Link to comment
tjb_altf4 Posted March 30, 2020 Share Posted March 30, 2020 2 hours ago, johner said: I wonder if someone could take the open source code of unraid and create a new ZFS 'unraid' config 🙂 ZFS is already an available plugin for Unraid. Quote Link to comment
JorgeB Posted March 30, 2020 Share Posted March 30, 2020 9 hours ago, johner said: unraid and create a new ZFS 'unraid' config 🙂 It would be the same, with Unraid each array disk is a separate filesystem, same issues even if all array disks were using zfs. Quote Link to comment
johner Posted April 1, 2020 Share Posted April 1, 2020 (edited) I mean, apply the unraid algorithm into ZFS, e.g. pool create tank raidzunraid dev1 dev2 dev3 🙂 Having disk and fs integration would then allow for online bitrot detection and recovery. not sure how one would create a single pool/array but separate zfs filesystems per disk and still be able to mount a single disk if the array dies. Edited April 1, 2020 by johner typo Quote Link to comment
JorgeB Posted April 2, 2020 Share Posted April 2, 2020 I mean, apply the unraid algorithm into ZFS, e.g. pool create tank raidzunraid dev1 dev2 dev3 That would make it impossible to use different size disks or still access disks individually on a different system if needed, and you already have FreeNAS for that. not sure how one would create a single pool/array but separate zfs filesystems per disk and still be able to mount a single disk if the array dies. You can't create a zfs pool of multiple zfs filesystems. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.