How to impliment BTRFS Checksumming with unRAID?


Noob

Recommended Posts

Mind f-ing blown

 

;-)

 

BTW, yes, this does work with virtual disk images for VMs as well ;-).  And the time it takes to create a reflink copy is measured in seconds, even with a 35GB+ size vdisk.  What was even more surprising is that this worked, even though I disabled COW for my virtual disk image file.

 

So I was intrigued by this notion and decided to read into it and test it out. But according to the info on the archlinux wiki (https://wiki.archlinux.org/index.php/Btrfs#Copy-On-Write_.28CoW.29 in the tip) and a few other links coming up in google, it doesn't work with COW disabled, it requires COW to work.

I have tested this on my cache drive as well. When COW has been properly disabled I get this error message when trying to do a cp--reflink

cp: failed to clone ‘/mnt/cache/VM/2.fastq.gz’ from ‘Test.fatq.gz’: Invalid argument

It will still create a 0byte file with the name you tell it to create, but when you go to read the file with say zcat in my case it comes up empty. I have tested this on COW enabled folders and it works as per the example but I don't know how to see the amount of space it actually uses.

 

Am I missing some crucial step here?

Link to comment
  • 1 month later...
  • Replies 92
  • Created
  • Last Reply

Top Posters In This Topic

Any updates on this? I am currently debating whether I should go with btrfs or xfs with Dynamix File Integrity plugin.

 

You can't really compare BTRFS and the file integrity tool. One is not a substitute of the other.

 

Well, right, but I specifically meant for the hashing feature. In that sense they are comparable, aren't they? Especially since, as it stands, UnRAID does not implement the self healing aspect of btrfs. Just trying to better understand how btrfs is currently implemented in UnRAID.

Link to comment
Just trying to better understand how btrfs is currently implemented in UnRAID.
Opinions vary. Search the forums to see some of the reasoning, but the default setup right now is xfs for array data drives, and btrfs for multiple cache drive pooling. I personally use a single cache drive with xfs after a unmountable drive incident with my btrfs cache drive. If your hardware is perfect and your server never locks up or has an unplanned restart, btrfs seems fine. It just doesn't tolerate unclean shutdowns. The word I would use is fragile.
Link to comment

Just trying to better understand how btrfs is currently implemented in UnRAID.
Opinions vary. Search the forums to see some of the reasoning, but the default setup right now is xfs for array data drives, and btrfs for multiple cache drive pooling. I personally use a single cache drive with xfs after a unmountable drive incident with my btrfs cache drive. If your hardware is perfect and your server never locks up or has an unplanned restart, btrfs seems fine. It just doesn't tolerate unclean shutdowns. The word I would use is fragile.

 

Ok, that helps. Some of my drives are pretty old. Windows 10 seems to have issues with seeing some of them and I don't know why. They only show up in device manager if they are plugged in on start up and you hot swap them. They work fine in UnRAID. Not sure if it's software/driver or hardware related but this board has had some strange behavior over the years and I can't say I'd trust it with btrfs. Thanks for the input.

Link to comment
  • 1 month later...

Just discovered this thread, no idea how I missed this discussion!

 

On parity btrfs auto-healing, you would need BTRFS to provide a bit position, then be able to convert that to a physical bit position, then parity repair a limited bit range on that drive.  All doable I think, but needs several development tasks done by LimeTech, when they have time.

 

This is a little off-topic, but based on the discussion above about BTRFS repair, would I be correct in this evaluation?

 

Repair tools for a single disk formatted with -

* ReiserFS - very good

* XFS - weak

* BTRFS - none!

 

Repair tools for pooled disks -

* BTRFS - weak to fair?

 

From user reports of BTRFS issues and the lack of repair-ability, I'm finding it very hard to recommend BTRFS for a single disk (cache, array, or unassigned).  Only perhaps in a system that has proven to be extremely stable, never crashes, and has a UPS.

Link to comment
This is a little off-topic, but based on the discussion above about BTRFS repair, would I be correct in this evaluation?

 

Repair tools for a single disk formatted with -

* ReiserFS - very good

* XFS - weak

* BTRFS - none!

 

Repair tools for pooled disks -

* BTRFS - weak to fair?

 

From user reports of BTRFS issues and the lack of repair-ability, I'm finding it very hard to recommend BTRFS for a single disk (cache, array, or unassigned).  Only perhaps in a system that has proven to be extremely stable, never crashes, and has a UPS.

I can provide an anecdotal sample size of 1 on the repair of single disk BTRFS errors. I was fighting the V6 Reiserfs issue, and in the process ended up hard restarting the server several times. My BTRFS 120GB SSD survived several hard restarts, until it didn't. I am speculating that there were uncommitted writes on that restart, but in any case I ended up almost fully recovering my VM files, my windows VM was a little upset and did a checkdisk but survived.

 

I wouldn't call BTRFS single disk repair a totally lost cause, but documentation is sparse and conflicting, depending on what point in time it was written in the development cycle of BTRFS. In my case a rescue mount worked after discarding the journal, but it was nerve wracking to work through, it felt like I was forging new paths instead of following long trusted procedures.

 

I haven't had first hand experience with XFS repair (knock on wood) but I have had several repair opportunities with Reiser, and Reiser seems so bulletproof compared to BTRFS that I wouldn't consider a Reiser disk a lost cause until the drive is totally non-responsive. BTRFS, on the other hand, seems to give up if you look at it funny.

 

JonP with limetech has expressed the opinion that all BTRFS errors are hardware or user induced, that the base technology is solid, and I can't really disagree with that, however, the average unraid system is far from datacenter grade, and the average user is VERY far from a sysadmin.

Link to comment

JonP with limetech has expressed the opinion that all BTRFS errors are hardware or user induced, that the base technology is solid, and I can't really disagree with that, however, the average unraid system is far from datacenter grade, and the average user is VERY far from a sysadmin.

 

I've seen his and Tom's comments on BTRFS, and found them completely plausible.  I trust them, and I really want to lean that way, but user experience here in the field is not near as rosy.  And now that I have just learned how useless scrub is for single drives, it makes it really hard to recommend.  There's a btrfs check --repair command, but it has a terrible reputation, may cause more harm than good.  I can't recommend it at all, except for last ditch file recovery (NOT for repair).  For single drives, reformatting is the only repair available!

Link to comment

JonP with limetech has expressed the opinion that all BTRFS errors are hardware or user induced, that the base technology is solid, and I can't really disagree with that, however, the average unraid system is far from datacenter grade, and the average user is VERY far from a sysadmin.

 

I've seen his and Tom's comments on BTRFS, and found them completely plausible.  I trust them, and I really want to lean that way, but user experience here in the field is not near as rosy.  And now that I have just learned how useless scrub is for single drives, it makes it really hard to recommend.  There's a btrfs check --repair command, but it has a terrible reputation, may cause more harm than good.  I can't recommend it at all, except for last ditch file recovery (NOT for repair).  For single drives, reformatting is the only repair available!

 

Based on my research there are a lot of BTRFS file system tools, none of which are very well documented.

 

In Order of Recommended Trial:

Attempt to mount the faulty file system
1) mount -o recovery,ro {Attempts to mount the filesystem in recover read only mode}
1.5) mount -o degraded {Only for btrfs RAID5/RAID6 configurations}
2) btrfs-zero-log {can help mount a filesystem if the last blocks btrfs wrote before a crash were corrupted or out of order due to bad hardware or other bugs.}

Attempt to recover data from a file system that won't mount normally or with the steps above
1) btrfs restore {"a non-destructive method for attempting to recover data from an unmountable filesystem, does not attempt to repair the filesystem just extract data}
2) btrfs check --repair {Can only run on an unmounted filesystem, destructive will attempt to repair filesystem}

 

Scrub is something completely different and is only checking the metadata vs the metadata. It does not repair or fix anything.

 

I feel like a lot of the reasons people are having problems with repairing btrfs issues is because they go right to btrfs check... and or think scrub is the same thing as btrfs check.

Link to comment

I feel like a lot of the reasons people are having problems with repairing btrfs issues is because they go right to btrfs check... and or think scrub is the same thing as btrfs check.

The crux of the issue here is that when there is BTRFS corruption or file system issues of one sort or another, it seems the only way forward is to recover the data to another device, and reformat. This is NOT what should be happening with a filesystem IMHO. Most other file systems allow you to fix the issue in place, one way or another, and allow you to continue using it after the tools have given the FS a clean bill of health.

 

Are there any BTRFS tools that will consistently repair most filesystem issues of a single volume to a reusable state? When I had my issue, all I could do was get it mounted read only and copy my files. Nothing I tried would allow me to mount the drive normally and continue to use it.

Link to comment

I added a couple of 240gig Kingston SSD's into my system as a BTRFS cache array just over a month ago and so far they have been working well. It was just time to upgrade my cache drive since it was an old WD drive. As I have posted before, test your BTRFS drives to ensure they are stable. BTRFS does work for many people but it does causes issues for others. Overall, I doubt I'd try BTRFS for the array drives but in a RAID1 configuration it should be good.

 

I do agree that ReiserFS is a very robust filesystem despite it typically being shunned. It really was ahead of it's time, but the developers personal situation basically killed it as an acceptable filesystem.

Link to comment

I feel like a lot of the reasons people are having problems with repairing btrfs issues is because they go right to btrfs check... and or think scrub is the same thing as btrfs check.

The crux of the issue here is that when there is BTRFS corruption or file system issues of one sort or another, it seems the only way forward is to recover the data to another device, and reformat. This is NOT what should be happening with a filesystem IMHO. Most other file systems allow you to fix the issue in place, one way or another, and allow you to continue using it after the tools have given the FS a clean bill of health.

 

Are there any BTRFS tools that will consistently repair most filesystem issues of a single volume to a reusable state? When I had my issue, all I could do was get it mounted read only and copy my files. Nothing I tried would allow me to mount the drive normally and continue to use it.

 

I don't disagree. Having never had an issue forcing me to use any of these tools I can't really speak to how effective they are.

 

I do think an important thing to understand about BTRFS is that it's not like XFS or Rieserfs there isn't one single tool to run to repair the filesystem because btrfs is designed to be more self healing. Often what I see fixing BTRFS mount issues is the recovery mount option or the btrfs-zero-log tool.

 

That said I do want to understand what your process was exactly. Did you try mounting the file system with the recovery option?

 

Additional thought, mount recovery and mount -o recovery,ro might be throwing people off because that just seems like you are mounting the fautly file system not actually fixing it... but it's my understanding that doing the recovery mount will often fix the problems that prevented a clean mount and the btrfs partition will mount properly the next time. If that doesn't work it might be an issue that btrfs-zero-log can fix... and if that doesn't work, time to try restore or btrfs -check to a least copy the data.

 

IMO everyone is looking for a checkfs tool to fix btrfs because years and years of using checkfs tools have people thinking that's what you have to do.... But BTRFS is different...

 

Again I'm not an expert on any of this, all of this is based on my internet sleuthing and I have personally not had to try any of these things and can't vouch for them. If I sound accusatory or condescending that's not my goal, and I apologize... All I really want is a better understanding of what people tried that didn't work, to see if the process we are telling people to attempt is bad.

Link to comment

gundamguy, has LT documented this anywhere in the wiki?

 

Hint LT: it would be helpful if you were able to provide some basic steps / warnings about dealing with BTRFS and XFS even if that just means links to authoritative sources.

 

Oh yeah and why isn't there a link to the wiki at the top of every page of this forum?

Link to comment

Not that most here need it, but I wanted to clarify that we're talking about 2 modus operandi here, data recovery and drive repair.  A tool that helps with the first may be useless for the second.

 

For drive repair, the gold standard is a fresh format, produces a completely clean and trustable file system.  The only tool I know of (for the 3 file systems we work with) that can be trusted as equally as format is reiserfsck.  xfs_repair sometimes works and sometimes doesn't, and even when it appears to work, I'm not left with a great deal of confidence.  The user seems happy that it *looks* fine, so we leave it there.  And it may be fine, but we'll need a little more experience and few or no reports of strange behavior in an xfs_repair'd drive.  For BTRFS, I don't know of anything that even *claims* to completely repair a BTRFS format.  And if you can't be sure it worked, then there's no point in trying, because you HAVE to be able to trust your file system.  If I can't trust the result, then I am always going to want to reformat, so that I CAN trust it.

 

For data recovery, there are tools, but even those who write of them don't exude confidence in them.  In my research earlier, I saw btrfs restore, but rejected it completely, and I'm sorry I can't remember why.  I'm going to have to go back and reexamine it.  And I fully admit that it's possible I've jumped too quickly to conclusions.  Then I found btrfs check --repair, but the comments about it were somewhat negative, even admitting it could cause additional damage!  I saw btrfsck, but it was deprecated in favor of btrfs check --repair, which is about as poor a recommendation as you can get! That means it's worse than your worst tool.

 

I did NOT see the mount recovery options at all, and I don't know why (speaking of poorly documented!), as what I was reading was supposed to be all the tools available for repair or recovery.  I'll need to do more research on them.  But more than research, WE NEED ACTUAL EXPERIENCE, true stories we can trust.  ... And better documentation.

Link to comment

For data recovery, there are tools, but even those who write of them don't exude confidence in them.  In my research earlier, I saw btrfs restore, but rejected it completely, and I'm sorry I can't remember why.  I'm going to have to go back and reexamine it.  And I fully admit that it's possible I've jumped too quickly to conclusions.  Then I found btrfs check --repair, but the comments about it were somewhat negative, even admitting it could cause additional damage!  I saw btrfsck, but it was deprecated in favor of btrfs check --repair, which is about as poor a recommendation as you can get! That means it's worse than your worst tool.

 

I did NOT see the mount recovery options at all, and I don't know why (speaking of poorly documented!), as what I was reading was supposed to be all the tools available for repair or recovery.  I'll need to do more research on them.  But more than research, WE NEED ACTUAL EXPERIENCE, true stories we can trust.  ... And better documentation.

 

Again you should take everything I'm about to say with a Grain of Salt because this is all from reading on the internet, not from personal or first hand experience.

 

The issue as I see it is that the way you actually repair a BTRFS filesystem is with mount recovery or mount recovery,ro or in rare cases btrfs-zero-log, btrfs restore is only to recover data from a filesystem that won't mount, it won't repair that filesystem. And the line of last resort is btrfs check --repair ... which as you noted is destructive.

 

I'm not saying BTRFS is great or anything. I'm just wondering if anyone has tried the recovery mount option or btrfs-zero-log option when they had issues, or did they go straight to btrfs check --repair?

Link to comment
  • 8 months later...

I'm checking back in after a year away from UnRAID. Now that 6.2 has dropped, can anyone update me on the status of BTRFS in unRAID and/or the ability to implement data integrity checksumming via a plugin? I found the Dynamix File Integrity thread, not sure whether that is what I want or not. It seems to "miss" lots of files based on the conversations going on over there. Any other plugins available?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.