Hi, I recently started my unRAID trial and setup my array using BTRFS. From what I've read it should be stable enough, and I do make backups of course! One of the big reasons I went with BTRFS over XFS is to protect against bitrot. However, I was talking with a user on Reddit that said the following: 

 

"BTRFS can identify errors in a RAID0 or 'no raid' scenario, but only RAID1 can correct. (and 5 and 6, but they're not ready)"

 

Is this true? Does BTRFS protect against bitrot when used on the array? Or does it only detect the errors but isn't able to correct them? Sorry if this has been documented before! I searched the forums but didn't find an answer to this. Any input would be much appreciated! Without the ability to fix bitrot errors I would probably choose XFS over BTRFS since it's been around much longer and seems to be very stable. 

15 minutes ago, rick9814911 said:

Or does it only detect the errors but isn't able to correct them?

 

Correct, since each data disk is a separate filesystem and using the default profile, btrfs will detect a checksum error but won't be able to fix it, that is what backups are for.

 

You could use dup profile for one or more disks, so data would be duplicated and any checksum error fixable, but obviously you'd lose half capacity on those disks.

9 minutes ago, johnnie.black said:

 

Correct, since each data disk is a separate filesystem and using the default profile, btrfs will detect a checksum error but won't be able to fix it, that is what backups are for.

 

You could use dup profile for one or more disks, so data would be duplicated and any checksum error fixable, but obviously you'd lose half capacity on those disks.

 

Thank you for clarifying! My only concern is that if I backup a file not knowing it's been corrupted, my backup will also be corrupt. That is the case right? 

 

Is there a recommended way to protect against bitrot on unRAID? I really love unRAID, but bitrot does worry me. 

Edited September 30, 20178 yr by rick9814911

Just now, rick9814911 said:

My only concern is that if I backup a file not knowing it's been corrupted, my backup will also be corrupt. That is the case right?

 

No, btrfs will error if it finds a checksum error, i.e., you won't be able to copy or play that file, you just need to check the log and will find a checksum error.

1 minute ago, johnnie.black said:

 

No, btrfs will error if it finds a checksum error, i.e., you won't be able to copy or play that file, you just need to check the log and will find a checksum error.

 

Excellent!! That's great news. Will it tell you which file is corrupt so I can restore it from the previous backup? 

4 minutes ago, rick9814911 said:

Will it tell you which file is corrupt so I can restore it from the previous backup? 

 

Yes, file name will appear on the syslog, any time you want you can also run a scrub to check that everything is OK.

1 minute ago, johnnie.black said:

 

Yes, file name will appear on the syslog, any time you want you can also run a scrub to check that everything is OK.

 

Thank you so much! You've been a huge help. 

So would a scrub not potentially fix the problem? Or is it not the sort of problem that is fixable? Is the only choice restoring the corrupted file?

9 minutes ago, ashman70 said:

So would a scrub not potentially fix the problem?

 

A scrub can only fix checksum errors on a redundant btrfs filesystem, e.g. a raid1 cache pool.

On a speculative note, how common would you say Bitrot is? People talk about it but I've never heard of anyone experiencing it, which of course doesn't mean anything.

4 minutes ago, ashman70 said:

On a speculative note, how common would you say Bitrot is?

 

Never seen it and I would guess no very common, but I did have a couple of situations where checksums were very valuable, and that's why I want them, once when there were read errors on a second disk during a rebuild of a failed disk (this was before dual parity) and another time when a disk red-baled during a disk to disk move, checksums allowed me to quickly find the affect files and replaced them from backups.

so if Btrfs is used in an unraid array (with parity), when Btrfs raises the io error due to bitrot (if it does), wouldn't "unraid" then fix the file? isn't that the point of the unraid layer? I totally understand Btrfs can't in this scenario. given Btrfs redundancy.

7 hours ago, johner said:

wouldn't "unraid" then fix the file? isn't that the point of the unraid layer?

If you knew the problem was only on the actual disk and parity was still correct you could rebuild the disk to correct the error.

I did a bit of reading, so it looks like the unraid layer is below the filesystem (files view) layer, it works on the blocks. therefore, there would need to be a way for unraid to ask Btrfs what blocks the bad file resides on and then recover them.

 

I wonder if someone could take the open source code of unraid and create a new ZFS 'unraid' config 🙂

2 hours ago, johner said:

I wonder if someone could take the open source code of unraid and create a new ZFS 'unraid' config 🙂

ZFS is already an available plugin for Unraid.

 

9 hours ago, johner said:

unraid and create a new ZFS 'unraid' config 🙂

It would be the same, with Unraid each array disk is a separate filesystem, same issues even if all array disks were using zfs.

I mean, apply the unraid algorithm into ZFS, e.g. pool create tank raidzunraid dev1 dev2 dev3 🙂

 

Having disk and fs integration would then allow for online bitrot detection and recovery. not sure how one would create a single pool/array but separate zfs filesystems per disk and still be able to mount a single disk if the array dies.

Edited April 1, 20206 yr by johner
typo

I mean, apply the unraid algorithm into ZFS, e.g. pool create tank raidzunraid dev1 dev2 dev3 [emoji846]

That would make it impossible to use different size disks or still access disks individually on a different system if needed, and you already have FreeNAS for that.

 

not sure how one would create a single pool/array but separate zfs filesystems per disk and still be able to mount a single disk if the array dies.

You can't create a zfs pool of multiple zfs filesystems.