Jump to content
jonp

Scrub BTRFS filesystems for validation.

10 posts in this topic Last Reply

Recommended Posts

I migrated to unRAID from FreeBSD and the thing I miss most of all from FreeBSD is the ability to scrub.  I am probably going to convert all my drives to BTRFS primarily to gain the ability to perform scrubs again.  If this were integrated into the GUI it would make me very happy.  I think others will find the value in performing scrubs once they have the capability. 

 

It is surprising how much bit rot happens on large arrays.  I usually found errors during zpool scrubs about once every few months.  I have had BDrips that had become unplayable due to rot be perfectly restored through zpool scrubs.

 

This is the number one feature I am looking forward to.

 

craigr

Share this post


Link to post

I will migrate from reiserfs when this feature is implemented.  I've definitely been experiencing bitrot over the years.  It's less noticeable now that I have most tv shows replaced with blurays as they're released, but a lot of older stuff that hasn't been re-issued is riddled with pops and blips during playback that wasn't there originally.

 

This feature will really kick it up a notch. 

 

Jon, If this gets implemented, I assume it will make btrfs the preferred FS?

Share this post


Link to post

This feature will really kick it up a notch. 

 

Jon, If this gets implemented, I assume it will make btrfs the preferred FS?

 

I'm with you on this.

In fact I'm working on two programs at the current to help track and possibly fix bitrot.

 

The first is a sqlite database of all files and hashes.

You can track that here.

RFC: MD5 checksum/Hash Software

http://lime-technology.com/forum/index.php?topic=34988.msg325400#msg325400

 

The second piece I'm considering is a par2 folder sweeper that will sweep down a directories tree and put a set of par2 files in each directory.

Capable of determining a problem and possibly??!?? repairing it.

 

The BTRFS checksum will only tell you an inode does not computer the checksum. I think it can only repair it from a snapshot or mirror.

 

Anyway finding the file that the inode references requires a find down the whole tree. so the sqlocate db I'm working on will allow a quicker query if the disk has been cataloged.  In addition that tool will allow you to locate and/or export an md5sum file for added assurance

 

I think with all of these items at our disposal, it could go a long way to safeguarding our data.

Share this post


Link to post

This was added in -beta7.

 

You sneaky bugger! :D  If that's the case, why aren't you guys making a bigger deal about using BTRFS?  Scrubbing is a huge feature win for unriad.

Share this post


Link to post

I migrated to unRAID from FreeBSD and the thing I miss most of all from FreeBSD is the ability to scrub.  I am probably going to convert all my drives to BTRFS primarily to gain the ability to perform scrubs again.  If this were integrated into the GUI it would make me very happy.  I think others will find the value in performing scrubs once they have the capability. 

 

It is surprising how much bit rot happens on large arrays.  I usually found errors during zpool scrubs about once every few months.  I have had BDrips that had become unplayable due to rot be perfectly restored through zpool scrubs.

 

This is the number one feature I am looking forward to.

 

craigr

 

how can i get my drives easily converted to btrfs ? one at a time ?

 

is it worth me to do for the scrubbing ?

 

thanks!

Share this post


Link to post

I migrated to unRAID from FreeBSD and the thing I miss most of all from FreeBSD is the ability to scrub.  I am probably going to convert all my drives to BTRFS primarily to gain the ability to perform scrubs again.  If this were integrated into the GUI it would make me very happy.  I think others will find the value in performing scrubs once they have the capability. 

 

It is surprising how much bit rot happens on large arrays.  I usually found errors during zpool scrubs about once every few months.  I have had BDrips that had become unplayable due to rot be perfectly restored through zpool scrubs.

 

This is the number one feature I am looking forward to.

 

craigr

 

how can i get my drives easily converted to btrfs ? one at a time ?

 

is it worth me to do for the scrubbing ?

 

thanks!

 

You'll have to empty a drive of all files, then convert it put the files back, then move on to the next drive doing the same thing.

Share this post


Link to post

I migrated to unRAID from FreeBSD and the thing I miss most of all from FreeBSD is the ability to scrub.  I am probably going to convert all my drives to BTRFS primarily to gain the ability to perform scrubs again.  If this were integrated into the GUI it would make me very happy.  I think others will find the value in performing scrubs once they have the capability. 

 

It is surprising how much bit rot happens on large arrays.  I usually found errors during zpool scrubs about once every few months.  I have had BDrips that had become unplayable due to rot be perfectly restored through zpool scrubs.

 

This is the number one feature I am looking forward to.

 

craigr

 

how can i get my drives easily converted to btrfs ? one at a time ?

 

is it worth me to do for the scrubbing ?

 

thanks!

 

You'll have to empty a drive of all files, then convert it put the files back, then move on to the next drive doing the same thing.

Technically you don't have to empty the drive, formatting it with a different file system will erase everything on it quite nicely. You do however, need to make absolutely sure that you have a backup (or two) of the contents of the drive you are converting, because it will be erased.

 

I have no good comment on whether it's a good idea to convert at this time, I've seen more unexplained data loss on these forums with btrfs cache drives in the small amount of time it's been available than I think I've ever seen with reiserfs. I'm not sure btrfs is quite ready for prime time yet.

 

That said, my server has a btrfs cache drive, and it's been fine so far.

Share this post


Link to post

Is there documentation on how to use the scrubbing feature?  I'll upgrade to BETA to start using this feature.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.