Can BTRFS fix bitrot?


Recommended Posts

Hi, I recently started my unRAID trial and setup my array using BTRFS. From what I've read it should be stable enough, and I do make backups of course! One of the big reasons I went with BTRFS over XFS is to protect against bitrot. However, I was talking with a user on Reddit that said the following: 

 

"BTRFS can identify errors in a RAID0 or 'no raid' scenario, but only RAID1 can correct. (and 5 and 6, but they're not ready)"

 

Is this true? Does BTRFS protect against bitrot when used on the array? Or does it only detect the errors but isn't able to correct them? Sorry if this has been documented before! I searched the forums but didn't find an answer to this. Any input would be much appreciated! Without the ability to fix bitrot errors I would probably choose XFS over BTRFS since it's been around much longer and seems to be very stable. 

  • Like 1
Link to comment
15 minutes ago, rick9814911 said:

Or does it only detect the errors but isn't able to correct them?

 

Correct, since each data disk is a separate filesystem and using the default profile, btrfs will detect a checksum error but won't be able to fix it, that is what backups are for.

 

You could use dup profile for one or more disks, so data would be duplicated and any checksum error fixable, but obviously you'd lose half capacity on those disks.

Link to comment
9 minutes ago, johnnie.black said:

 

Correct, since each data disk is a separate filesystem and using the default profile, btrfs will detect a checksum error but won't be able to fix it, that is what backups are for.

 

You could use dup profile for one or more disks, so data would be duplicated and any checksum error fixable, but obviously you'd lose half capacity on those disks.

 

Thank you for clarifying! My only concern is that if I backup a file not knowing it's been corrupted, my backup will also be corrupt. That is the case right? 

 

Is there a recommended way to protect against bitrot on unRAID? I really love unRAID, but bitrot does worry me. 

Edited by rick9814911
Link to comment
Just now, rick9814911 said:

My only concern is that if I backup a file not knowing it's been corrupted, my backup will also be corrupt. That is the case right?

 

No, btrfs will error if it finds a checksum error, i.e., you won't be able to copy or play that file, you just need to check the log and will find a checksum error.

Link to comment
1 minute ago, johnnie.black said:

 

No, btrfs will error if it finds a checksum error, i.e., you won't be able to copy or play that file, you just need to check the log and will find a checksum error.

 

Excellent!! That's great news. Will it tell you which file is corrupt so I can restore it from the previous backup? 

Link to comment
4 minutes ago, ashman70 said:

On a speculative note, how common would you say Bitrot is?

 

Never seen it and I would guess no very common, but I did have a couple of situations where checksums were very valuable, and that's why I want them, once when there were read errors on a second disk during a rebuild of a failed disk (this was before dual parity) and another time when a disk red-baled during a disk to disk move, checksums allowed me to quickly find the affect files and replaced them from backups.

  • Like 1
Link to comment
  • 2 years later...

so if Btrfs is used in an unraid array (with parity), when Btrfs raises the io error due to bitrot (if it does), wouldn't "unraid" then fix the file? isn't that the point of the unraid layer? I totally understand Btrfs can't in this scenario. given Btrfs redundancy.

Link to comment

I did a bit of reading, so it looks like the unraid layer is below the filesystem (files view) layer, it works on the blocks. therefore, there would need to be a way for unraid to ask Btrfs what blocks the bad file resides on and then recover them.

 

I wonder if someone could take the open source code of unraid and create a new ZFS 'unraid' config 🙂

  • Like 1
Link to comment

I mean, apply the unraid algorithm into ZFS, e.g. pool create tank raidzunraid dev1 dev2 dev3 🙂

 

Having disk and fs integration would then allow for online bitrot detection and recovery. not sure how one would create a single pool/array but separate zfs filesystems per disk and still be able to mount a single disk if the array dies.

Edited by johner
typo
Link to comment
I mean, apply the unraid algorithm into ZFS, e.g. pool create tank raidzunraid dev1 dev2 dev3

That would make it impossible to use different size disks or still access disks individually on a different system if needed, and you already have FreeNAS for that.

 

not sure how one would create a single pool/array but separate zfs filesystems per disk and still be able to mount a single disk if the array dies.

You can't create a zfs pool of multiple zfs filesystems.

 

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.