Default to BTRFS? Compression?

[NLS]

I would like to (re)open this discussion, hopefully people that have already done it will chime in and tell us what is positive and negative in this move.

 

So to organize this a bit:
 

1) Official docs, claim that BTRFS is not as robust as XFS yet. But they use BTRFS by default in cache (where the risk is considered smaller? Yet most people's VMs are there). Is this claim still valid in Oct 22? Or should UNRAID move to default BTRFS "soonish"?

 

2) Compression, while default in most "competing" solutions that use ZFS, is not even a (GUI) option in UNRAID. People that use it (I believe there are a few), feel it should be an option that can be set "officially"? (or even default)

 

3) Recovery options when moving to BTRFS or moving to BTRFS with compression. Are they available? Is it the same as with XFS?

 

4) Load on CPU? Actual saving benefits? Some people claim that for certain uses, writing with compression (or reading compressed) can even speed up things, as less data are read from the actual slow media (or writen to it) and the load moves to the CPU. In practice?
 

5) For people that have actually done it: Can someone give a streamlined stupid-proof process on how to achieve BTRFS with compression in UNRAID?

 

[ChatNoir]

35 minutes ago, NLS said:

1) Official docs, claim that BTRFS is not as robust as XFS yet. But they use BTRFS by default in cache (where the risk is considered smaller? Yet most people's VMs are there). Is this claim still valid in Oct 22? Or should UNRAID move to default BTRFS "soonish"?

Limetech position seems pretty reasonable to me if you consider all aspect (ease of use, robustness, etc.).

XFS is more tolerant to external issues, as such, it is the default for the Array.

But BTRFS RAID profiles allow to pool drives, it is an understandable choice to use it as default for Pools. Even for one drive pool as it will allow the user to add additional drives to a Pool in the future without redoing the Pool entirely.

 

49 minutes ago, NLS said:

2) Compression, while default in most "competing" solutions that use ZFS, is not even a (GUI) option in UNRAID. People that use it (I believe there are a few), feel it should be an option that can be set "officially"? (or even default)

Limetech is working for quite some time on the ZFS implementation as the 6.11 release note confirms. It will come.

 

Quote

 

The 6.11 release includes bug fixes, update of base packages, update to 5.19.x Linux kernel, and minor feature improvements.

 

Sorry no major new feature but instead we are paying some "technical debt" and laying the groundwork necessary to add better third-party driver and ZFS support.  Although, Samba is updated to version 4.17 and we're seeing some significant performance increases.

 

 

[NLS]

6 minutes ago, ChatNoir said:

Limetech position seems pretty reasonable to me if you consider all aspect (ease of use, robustness, etc.).

XFS is more tolerant to external issues, as such, it is the default for the Array.

But BTRFS RAID profiles allow to pool drives, it is an understandable choice to use it as default for Pools. Even for one drive pool as it will allow the user to add additional drives to a Pool in the future without redoing the Pool entirely.

 

Limetech is working for quite some time on the ZFS implementation as the 6.11 release note confirms. It will come.

 

 

 

For #1. But is it? I mean BTRFS handles bit-rot and (about the tolerance to external issues) is CoW.

 

For #2. That is GREAT news and seems I didn't notice in the latest release notes. I hope they are actively working on this.

 

Thanks for you reply, I still need actual users that have tried BTRFS on array and even better, compression, to chime in. I am 100% sure we can find at least one.

 

[gilahacker]

On 10/10/2022 at 2:51 AM, NLS said:

I still need actual users that have tried BTRFS on array and even better, compression, to chime in. I am 100% sure we can find at least one.

I formatted a single 4 TB drive in my array as BTRFS and enabled compression using `chattr -c`.

I also installed the "compsize" tool, to show me what kind of compression I was getting:

Processed 213570 files, 1597070 regular extents (1597070 refs), 46817 inline, 223456 fragments.
Type       Perc     Disk Usage   Uncompressed Referenced
TOTAL       96%      1.8T         1.8T         1.8T
none       100%      1.7T         1.7T         1.7T
zlib        43%       50G         115G         115G

Out of 1.8 TB total data, only 115 GB was compressed. It was getting a good compression ratio (2.3:1).

I wanted to increase the compression level, but couldn't find any way to do that since there's no entry for the drive in /etc/fstab like there would be on a non-unRAID setup. While reading up on that, I discovered zstd (even though it's been around for a while now) and that seemed like a better option than the default zlib, so I ran btrfs filesystem defragment -rf -czstd /mnt/disk18 to switch to zstd and it compressed a lot more of the files:

Processed 149606 files, 12382461 regular extents (12382461 refs), 29274 inline, 781994 fragments.
Type       Perc     Disk Usage   Uncompressed Referenced
TOTAL       73%      1.2T         1.7T         1.7T
none       100%      405G         405G         405G
zstd        65%      915G         1.3T         1.3T

(note that I deleted some unnecessary files after the defragment, so my total size decreased)

The compression ratio dropped (1.45:1), but >10x more data is being compressed now so I'm "saving" ~416 GB of disk space whereas I was only saving 65GB before.

Based on what I've found, zstd compression should be as good or better than zlib while being a whole lot faster. Increasing the compression level means it takes longer to initially compress, but decompression rate stays about the same (a trait zlib shares). I'd love to be able to crank up the compression level, but the changes to allow that either aren't merged into the btrfs command line tool yet or the version we have isn't up to date. I found lots of discussion about adding them on github, but haven't compared version numbers or anything.

There's also supposedly a way to force compression on all files as, by default, btrfs only tries to compress the beginning of the file and if it doesn't appear to compress well it just leaves the whole file uncompressed. This is why even after running the defrag command above, 405 GB is still listed as having no compression. I haven't figured out if there's a way to do that yet.

In my particular case, many of the files are already RAR or ZIP or whatever, which probably wouldn't compress well anyways, but I've been slowly crawling through and unpacking those files so I can actually browse their contents in my file manager. This drive contains my collection of STL files for 3D printing which take up the majority of the space and a few old system backups that take up 158 GB.

[NLS]

Thank you! Very interesting.

So it is probably a viable solution to get more space.
Hope it gets evaluated by UNRAID team and gets actual GUI support.

 

[JorgeB]

13 hours ago, gilahacker said:

I wanted to increase the compression level, but couldn't find any way to do that since there's no entry for the drive in /etc/fstab

You can use the remount option, e.g. have a user script after array start:

 

mount -o remount -o compress=zstd:10 /mnt/cache

 

P.S. I've asked LT to enable compression by default when zfs is available and Tom suggested he might do the same for btrfs, so soon there might be two filesystem compression options.

 

[NLS]

good to see the cogs moving

 

[primeval_god]

You can also enable compression on a subvolume, with more options than chattr but less that with mount. Anything written under the subvolume, including other subvolumes are compressed as well.